[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:34 dist-test-slave-fgb3 krb5kdc[2013](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:34 dist-test-slave-fgb3 krb5kdc[2013](info): set up 2 sockets
Aug 06 18:44:34 dist-test-slave-fgb3 krb5kdc[2013](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 18:44:35.497107 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:35 dist-test-slave-fgb3 krb5kdc[2013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:35.626804 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:39701
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:38209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:39701
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:35.742986 2031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:35.743247 2031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:35.743305 2031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:35.747237 2031 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:35.747290 2031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:35.747321 2031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:35.747344 2031 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:35.747371 2031 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:35.752561 2031 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:38209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:39701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:39701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2031
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:35.753612 2031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:35.754693 2031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:44:35.765070 2031 server_base.cc:1034] running on GCE node
W20260806 18:44:35.765147 2042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:35.765398 2038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:35.765486 2037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:35.766112 2031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:35.767331 2031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:35.768503 2031 hybrid_clock.cc:648] HybridClock initialized: now 1786041875768479 us; error 39 us; skew 500 ppm
Aug 06 18:44:35 dist-test-slave-fgb3 krb5kdc[2013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041875, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:35.771226 2031 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:35.772128 2031 webserver.cc:460] Webserver started at http://127.1.245.126:38549/ using document root <none> and password file <none>
I20260806 18:44:35.772706 2031 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:35.772771 2031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:35.772969 2031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:35.775189 2031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "e9624c5b918b4826b3aa1935117e1481"
format_stamp: "Formatted at 2026-08-06 18:44:35 on dist-test-slave-fgb3"
server_key: "7f96ea2ae00d36e8d7050fb6aa9e2627"
server_key_iv: "7ebd90ced8461f65bc28d15e686df2f6"
server_key_version: "encryptionkey@0"
I20260806 18:44:35.775782 2031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "e9624c5b918b4826b3aa1935117e1481"
format_stamp: "Formatted at 2026-08-06 18:44:35 on dist-test-slave-fgb3"
server_key: "7f96ea2ae00d36e8d7050fb6aa9e2627"
server_key_iv: "7ebd90ced8461f65bc28d15e686df2f6"
server_key_version: "encryptionkey@0"
I20260806 18:44:35.779726 2031 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:35.782222 2054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:35.783531 2031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:44:35.783653 2031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "e9624c5b918b4826b3aa1935117e1481"
format_stamp: "Formatted at 2026-08-06 18:44:35 on dist-test-slave-fgb3"
server_key: "7f96ea2ae00d36e8d7050fb6aa9e2627"
server_key_iv: "7ebd90ced8461f65bc28d15e686df2f6"
server_key_version: "encryptionkey@0"
I20260806 18:44:35.783752 2031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:35.800158 2031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:35.803563 2031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:35.803752 2031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:35.811079 2031 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:39701
I20260806 18:44:35.811121 2121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:39701 every 8 connection(s)
I20260806 18:44:35.812105 2031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:35.815017 2122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:35.816330 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2031
I20260806 18:44:35.816497 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:35.818234 2005 external_mini_cluster.cc:1428] Setting key 55bcc000ca271cc2fd2f259c80b40c0d
I20260806 18:44:35.822266 2122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481: Bootstrap starting.
I20260806 18:44:35.824620 2122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:35.825438 2122 log.cc:824] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:35.828017 2122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481: No bootstrap required, opened a new log
I20260806 18:44:35.831502 2122 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9624c5b918b4826b3aa1935117e1481" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39701 } }
I20260806 18:44:35.831774 2122 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:35.831858 2122 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9624c5b918b4826b3aa1935117e1481, State: Initialized, Role: FOLLOWER
I20260806 18:44:35.832384 2122 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9624c5b918b4826b3aa1935117e1481" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39701 } }
I20260806 18:44:35.832566 2122 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:35.832638 2122 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:35.832751 2122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:35.833859 2122 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9624c5b918b4826b3aa1935117e1481" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39701 } }
I20260806 18:44:35.834205 2122 leader_election.cc:302] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9624c5b918b4826b3aa1935117e1481; no voters:
I20260806 18:44:35.834556 2122 leader_election.cc:288] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:35.834682 2133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:35.835000 2133 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [term 1 LEADER]: Becoming Leader. State: Replica: e9624c5b918b4826b3aa1935117e1481, State: Running, Role: LEADER
I20260806 18:44:35.835407 2133 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9624c5b918b4826b3aa1935117e1481" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39701 } }
I20260806 18:44:35.835887 2122 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:35.837416 2135 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9624c5b918b4826b3aa1935117e1481. Latest consensus state: current_term: 1 leader_uuid: "e9624c5b918b4826b3aa1935117e1481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9624c5b918b4826b3aa1935117e1481" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39701 } } }
I20260806 18:44:35.837561 2135 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:35.837939 2134 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9624c5b918b4826b3aa1935117e1481" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9624c5b918b4826b3aa1935117e1481" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39701 } } }
I20260806 18:44:35.838070 2134 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:35.838486 2150 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 18:44:35 dist-test-slave-fgb3 krb5kdc[2013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:35.842244 2150 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:35.848294 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:35.822232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56006 (local address 127.1.245.126:39701)
0806 18:44:35.822761 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:35.822780 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:35.823829 (+ 1049us) server_negotiation.cc:407] Connection header received
0806 18:44:35.827408 (+ 3579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:35.827432 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:35.827844 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:35.828230 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:35.830377 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:35.833276 (+ 2899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:35.834857 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:35.835543 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:35.835700 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:35.841561 (+ 5861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:35.841569 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:35.841585 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:35.841622 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:35.843902 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:35.844652 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:35.844659 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:35.844664 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:35.844829 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:35.845254 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:35.845257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:35.845259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:35.845789 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:35.845995 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:35.847046 (+ 1051us) server_negotiation.cc:299] Negotiation successful
0806 18:44:35.847359 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":156,"threads_started":1}
I20260806 18:44:35.848420 2150 catalog_manager.cc:1347] Generated new cluster ID: 8013aa8a66fd47fcb19a9f86d3527d45
I20260806 18:44:35.848822 2150 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:35.858693 2150 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:35.859668 2150 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:35.864048 2150 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9624c5b918b4826b3aa1935117e1481: Generated new TSK 0
I20260806 18:44:35.864842 2150 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:44:35.897339 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39701
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:38209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:36.028373 2169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:36.028644 2169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:36.028689 2169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:36.032850 2169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:36.032909 2169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:36.032968 2169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:44:36.039057 2169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:38209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39701
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:36.040105 2169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:36.041088 2169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:36.050940 2178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:36.051010 2176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:36.051021 2175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:36.051337 2169 server_base.cc:1034] running on GCE node
I20260806 18:44:36.051721 2169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:36.052280 2169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:36.053438 2169 hybrid_clock.cc:648] HybridClock initialized: now 1786041876053417 us; error 29 us; skew 500 ppm
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:36.056648 2169 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:44:36.057533 2169 webserver.cc:460] Webserver started at http://127.1.245.65:42381/ using document root <none> and password file <none>
I20260806 18:44:36.058070 2169 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:36.058138 2169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:36.058369 2169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:36.060367 2169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "565cd25a06f14f8182649609649fe7be"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "2e613e79c29a4907130479f90859968f"
server_key_iv: "61d617f81ec9609ef62b8466aa1c1a74"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.060976 2169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "565cd25a06f14f8182649609649fe7be"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "2e613e79c29a4907130479f90859968f"
server_key_iv: "61d617f81ec9609ef62b8466aa1c1a74"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.064595 2169 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260806 18:44:36.066933 2187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.068063 2169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:44:36.068171 2169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "565cd25a06f14f8182649609649fe7be"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "2e613e79c29a4907130479f90859968f"
server_key_iv: "61d617f81ec9609ef62b8466aa1c1a74"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.068262 2169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:36.079648 2169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:36.082723 2169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:36.082898 2169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:36.083482 2169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:36.084451 2169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:36.084501 2169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.084554 2169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:36.084582 2169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.102371 2169 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:45207
I20260806 18:44:36.102547 2338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:45207 every 8 connection(s)
I20260806 18:44:36.103385 2169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:44:36.104426 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2169
I20260806 18:44:36.104543 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:44:36.104818 2005 external_mini_cluster.cc:1428] Setting key 044b1453e8b0632d392e53d32273bca5
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:44:36.120242 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:36.105703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:41897 (local address 127.1.245.126:39701)
0806 18:44:36.105863 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:36.105868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:36.106935 (+ 1067us) server_negotiation.cc:407] Connection header received
0806 18:44:36.109084 (+ 2149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:36.109088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:36.109147 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:36.109236 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:36.110803 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.111349 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.113389 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.113647 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.113738 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:36.117056 (+ 3318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:36.117061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:36.117063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:36.117093 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:36.118307 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.118791 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.118793 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.118794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.118869 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.119148 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.119150 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.119152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.119342 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:36.119419 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:36.119965 (+ 546us) server_negotiation.cc:299] Negotiation successful
0806 18:44:36.120111 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:44:36.121163 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:36.106162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:39701 (local address 127.1.245.65:41897)
0806 18:44:36.106743 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:36.106818 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:36.108859 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:36.109436 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:36.109443 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:36.109948 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:36.110642 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:36.110661 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.111477 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.111480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:36.113255 (+ 1775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:36.113269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.113819 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.113824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:36.114118 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:36.114849 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:36.114874 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:36.116861 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:36.118446 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:36.118451 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:36.118464 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:36.118710 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:36.118936 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:36.118938 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:36.118939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:36.119064 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:36.119425 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:36.119432 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:36.119675 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 18:44:36.119872 (+ 197us) client_negotiation.cc:240] Negotiation successful
0806 18:44:36.120249 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":156,"threads_started":1}
I20260806 18:44:36.122633 2339 heartbeater.cc:347] Connected to a master server at 127.1.245.126:39701
I20260806 18:44:36.122926 2339 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:36.123504 2339 heartbeater.cc:511] Master 127.1.245.126:39701 requested a full tablet report, sending...
I20260806 18:44:36.125149 2077 ts_manager.cc:194] Registered new tserver with Master: 565cd25a06f14f8182649609649fe7be (127.1.245.65:45207)
I20260806 18:44:36.126286 2077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:41897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:44:36.137681 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39701
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:38209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:36.261934 2345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:36.262188 2345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:36.262233 2345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:36.266388 2345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:36.266448 2345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:36.266521 2345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:44:36.271838 2345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:38209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39701
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:36.272925 2345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:36.273947 2345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:36.283973 2356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:36.283984 2358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:36.284179 2355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:36.284224 2345 server_base.cc:1034] running on GCE node
I20260806 18:44:36.284617 2345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:36.285244 2345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:36.286383 2345 hybrid_clock.cc:648] HybridClock initialized: now 1786041876286359 us; error 38 us; skew 500 ppm
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:36.289084 2345 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:44:36.290047 2345 webserver.cc:460] Webserver started at http://127.1.245.66:37203/ using document root <none> and password file <none>
I20260806 18:44:36.290702 2345 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:36.290778 2345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:36.290954 2345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:36.292948 2345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "eec04121188b48eb8ffec1366684b35d"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "20bdd3df65c67f4759bf4242a0f153e2"
server_key_iv: "8ed8802b84eb8758e7ae094444802cf3"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.293529 2345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "eec04121188b48eb8ffec1366684b35d"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "20bdd3df65c67f4759bf4242a0f153e2"
server_key_iv: "8ed8802b84eb8758e7ae094444802cf3"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.297797 2345 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:44:36.300400 2367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.301636 2345 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:36.301741 2345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "eec04121188b48eb8ffec1366684b35d"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "20bdd3df65c67f4759bf4242a0f153e2"
server_key_iv: "8ed8802b84eb8758e7ae094444802cf3"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.301851 2345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:36.321976 2345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:36.325090 2345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:36.325256 2345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:36.325840 2345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:36.326735 2345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:36.326786 2345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.326826 2345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:36.326846 2345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.335578 2345 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:39109
I20260806 18:44:36.335621 2529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:39109 every 8 connection(s)
I20260806 18:44:36.336534 2345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:44:36.345078 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2345
I20260806 18:44:36.345186 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:44:36.345441 2005 external_mini_cluster.cc:1428] Setting key 0a97f9f54fec556d739568688adb79c8
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:36.350494 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:36.338642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:41503 (local address 127.1.245.126:39701)
0806 18:44:36.338778 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:36.338782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:36.339687 (+ 905us) server_negotiation.cc:407] Connection header received
0806 18:44:36.340972 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:36.340975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:36.341024 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:36.341094 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:36.342186 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.342773 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.343952 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.344153 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.344220 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:36.346819 (+ 2599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:36.346824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:36.346827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:36.346855 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:36.348383 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.349008 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.349011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.349012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.349088 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.349416 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.349420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.349422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.349602 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:36.349687 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:36.350175 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 18:44:36.350296 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:44:36.351320 2535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:36.339063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:39701 (local address 127.1.245.66:41503)
0806 18:44:36.339540 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:36.339583 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:36.340762 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:36.341230 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:36.341236 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:36.341591 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:36.342061 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:36.342074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.342889 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.342892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:36.343843 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:36.343853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.344306 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.344340 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:36.344494 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:36.345116 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:36.345132 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:36.346665 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:36.348530 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:36.348536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:36.348553 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:36.348920 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:36.349180 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:36.349183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:36.349184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:36.349321 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:36.349700 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:36.349707 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:36.349949 (+ 242us) client_negotiation.cc:769] Sending connection context
0806 18:44:36.350119 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:44:36.350483 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":124,"threads_started":1}
I20260806 18:44:36.352653 2532 heartbeater.cc:347] Connected to a master server at 127.1.245.126:39701
I20260806 18:44:36.352896 2532 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:36.353394 2532 heartbeater.cc:511] Master 127.1.245.126:39701 requested a full tablet report, sending...
I20260806 18:44:36.354591 2077 ts_manager.cc:194] Registered new tserver with Master: eec04121188b48eb8ffec1366684b35d (127.1.245.66:39109)
I20260806 18:44:36.355394 2077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:41503
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:44:36.375486 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39701
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:38209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:36.487470 2540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:36.487726 2540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:36.487771 2540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:36.491804 2540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:36.491863 2540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:36.491927 2540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:44:36.497138 2540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:38209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39701
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:36.498219 2540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:36.499260 2540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:36.508867 2547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:36.509008 2548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:36.508920 2551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:36.509095 2540 server_base.cc:1034] running on GCE node
I20260806 18:44:36.509712 2540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:36.510383 2540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:36.511559 2540 hybrid_clock.cc:648] HybridClock initialized: now 1786041876511542 us; error 29 us; skew 500 ppm
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:36.514418 2540 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:44:36.515347 2540 webserver.cc:460] Webserver started at http://127.1.245.67:41127/ using document root <none> and password file <none>
I20260806 18:44:36.515908 2540 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:36.515975 2540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:36.516177 2540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:36.518177 2540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "501d8764fdb94bf6b81efcc4eab8a1a4"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "6c2aaa26aeb7f7de9d0dd0f75d6a4967"
server_key_iv: "c7c63c48bac57e95fcd17b875d93233e"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.518759 2540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "501d8764fdb94bf6b81efcc4eab8a1a4"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "6c2aaa26aeb7f7de9d0dd0f75d6a4967"
server_key_iv: "c7c63c48bac57e95fcd17b875d93233e"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.522385 2540 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:44:36.524857 2563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.526062 2540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:44:36.526171 2540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "501d8764fdb94bf6b81efcc4eab8a1a4"
format_stamp: "Formatted at 2026-08-06 18:44:36 on dist-test-slave-fgb3"
server_key: "6c2aaa26aeb7f7de9d0dd0f75d6a4967"
server_key_iv: "c7c63c48bac57e95fcd17b875d93233e"
server_key_version: "encryptionkey@0"
I20260806 18:44:36.526265 2540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:36.541275 2540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:36.544287 2540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:36.544463 2540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:36.545028 2540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:36.545826 2540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:36.545874 2540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.545928 2540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:36.545953 2540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:36.554792 2540 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:40727
I20260806 18:44:36.554823 2715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:40727 every 8 connection(s)
I20260806 18:44:36.555652 2540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:44:36.562280 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2540
I20260806 18:44:36.562415 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:44:36.562659 2005 external_mini_cluster.cc:1428] Setting key 4600800c849dddf4b727fadd7740634d
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:36.569617 2125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:36.557616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:47787 (local address 127.1.245.126:39701)
0806 18:44:36.557777 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:36.557781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:36.558671 (+ 890us) server_negotiation.cc:407] Connection header received
0806 18:44:36.559999 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:36.560003 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:36.560061 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:36.560138 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:36.561392 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.561919 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.563158 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.563444 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.563532 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:36.566280 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:36.566283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:36.566285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:36.566338 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:36.567592 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.568099 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.568102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.568103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.568189 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.568579 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.568582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.568583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.568769 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:36.568851 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:36.569372 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 18:44:36.569483 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:44:36.570446 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:36.557978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:39701 (local address 127.1.245.67:47787)
0806 18:44:36.558529 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:36.558572 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:36.559791 (+ 1219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:36.560324 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:36.560331 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:36.560742 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:36.561251 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:36.561266 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.562057 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.562060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:36.563031 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:36.563041 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.563603 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.563607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:36.563805 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:36.564436 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:36.564456 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:36.566119 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:36.567703 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:36.567708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:36.567720 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:36.567995 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:36.568299 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:36.568302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:36.568304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:36.568478 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:36.568845 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:36.568850 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:36.569093 (+ 243us) client_negotiation.cc:769] Sending connection context
0806 18:44:36.569263 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:44:36.569544 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":118,"threads_started":1}
I20260806 18:44:36.571671 2716 heartbeater.cc:347] Connected to a master server at 127.1.245.126:39701
I20260806 18:44:36.571916 2716 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:36.572448 2716 heartbeater.cc:511] Master 127.1.245.126:39701 requested a full tablet report, sending...
I20260806 18:44:36.573486 2077 ts_manager.cc:194] Registered new tserver with Master: 501d8764fdb94bf6b81efcc4eab8a1a4 (127.1.245.67:40727)
I20260806 18:44:36.574033 2077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:47787
I20260806 18:44:36.577258 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:44:36.603927 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:36.594368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.1.245.65:45207)
0806 18:44:36.594647 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:36.594664 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:36.594693 (+ 29us) server_negotiation.cc:407] Connection header received
0806 18:44:36.594753 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:36.594760 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:36.594975 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:36.595117 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:36.595848 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.596633 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.597461 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.597836 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.597931 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:36.600361 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:36.600374 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:36.600390 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:36.600417 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:36.602013 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.602508 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.602514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.602515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.602600 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.602885 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.602889 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.602891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.603147 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:36.603377 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:36.603578 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 18:44:36.603734 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":64,"threads_started":1}
W20260806 18:44:36.604947 2226 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36144
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2013](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041876, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:44:36.631425 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:36.622988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.1.245.65:45207)
0806 18:44:36.623145 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:36.623149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:36.623268 (+ 119us) server_negotiation.cc:407] Connection header received
0806 18:44:36.623460 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:36.623464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:36.623511 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:36.623579 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:36.624288 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.624813 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.625635 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:36.625881 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:36.625972 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:36.628416 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:36.628419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:36.628421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:36.628446 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:36.629784 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.630166 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.630170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.630171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.630250 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:36.630552 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:36.630555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:36.630557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:36.630731 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:36.630849 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:36.631128 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 18:44:36.631250 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:44:36.632681 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2169
I20260806 18:44:36.649315 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2345
I20260806 18:44:36.670023 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2540
I20260806 18:44:36.681339 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2031
2026-08-06T18:44:36Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2381 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2735](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2735](info): set up 2 sockets
Aug 06 18:44:36 dist-test-slave-fgb3 krb5kdc[2735](info): commencing operation
krb5kdc: starting...
W20260806 18:44:37.818995 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:37 dist-test-slave-fgb3 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:37.945288 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:34817
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:34817
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:38.061756 2753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:38.062062 2753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:38.062129 2753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:38.066210 2753 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:38.066299 2753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:38.066354 2753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:38.066383 2753 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:38.066417 2753 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:38.071977 2753 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:34817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:34817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:38.073151 2753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:38.074247 2753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:38.083056 2762 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:38.083078 2759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:38.083122 2753 server_base.cc:1034] running on GCE node
W20260806 18:44:38.083078 2760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:38.083906 2753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:38.085059 2753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:38.086242 2753 hybrid_clock.cc:648] HybridClock initialized: now 1786041878086217 us; error 42 us; skew 500 ppm
Aug 06 18:44:38 dist-test-slave-fgb3 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:38.089120 2753 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:38.090152 2753 webserver.cc:460] Webserver started at http://127.1.245.126:41131/ using document root <none> and password file <none>
I20260806 18:44:38.090777 2753 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:38.090849 2753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:38.091066 2753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:38.093181 2753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "5f937dac0e5a40969257882300039a23"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "93460b7c151e89f22c6a2a592795bf54"
server_key_iv: "20934df5e320020e9231597735e76e91"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.093765 2753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "5f937dac0e5a40969257882300039a23"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "93460b7c151e89f22c6a2a592795bf54"
server_key_iv: "20934df5e320020e9231597735e76e91"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.097615 2753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:44:38.100224 2774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.101493 2753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260806 18:44:38.101606 2753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "5f937dac0e5a40969257882300039a23"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "93460b7c151e89f22c6a2a592795bf54"
server_key_iv: "20934df5e320020e9231597735e76e91"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.101720 2753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:38.114734 2753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:38.118010 2753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:38.118172 2753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:38.125646 2753 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:34817
I20260806 18:44:38.125666 2843 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:34817 every 8 connection(s)
I20260806 18:44:38.126667 2753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:38.129487 2844 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:38.132807 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2753
I20260806 18:44:38.132905 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:38.133148 2005 external_mini_cluster.cc:1428] Setting key b96c21563f34a3d8064000730dbf957e
I20260806 18:44:38.135571 2844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23: Bootstrap starting.
I20260806 18:44:38.137851 2844 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:38.138662 2844 log.cc:824] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:38.141042 2844 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23: No bootstrap required, opened a new log
Aug 06 18:44:38 dist-test-slave-fgb3 krb5kdc[2735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:38.145665 2844 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f937dac0e5a40969257882300039a23" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34817 } }
I20260806 18:44:38.145937 2844 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:38.145999 2844 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f937dac0e5a40969257882300039a23, State: Initialized, Role: FOLLOWER
I20260806 18:44:38.146526 2844 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f937dac0e5a40969257882300039a23" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34817 } }
I20260806 18:44:38.146700 2844 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:38.146775 2844 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:38.146871 2844 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:38.147809 2844 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f937dac0e5a40969257882300039a23" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34817 } }
I20260806 18:44:38.148062 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.134653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35468 (local address 127.1.245.126:34817)
0806 18:44:38.135130 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:38.135146 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:38.135183 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:44:38.135863 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:38.135883 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:38.136218 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:38.136535 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:38.137336 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.138433 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.139437 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.140151 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.140324 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.143018 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:38.143025 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:38.143043 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:38.143075 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:38.145018 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.145455 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.145461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.145467 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.145608 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.145876 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.145879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.145881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.146438 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:38.146644 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:38.146991 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 18:44:38.147261 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":148,"threads_started":1}
I20260806 18:44:38.148137 2844 leader_election.cc:302] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5f937dac0e5a40969257882300039a23; no voters:
I20260806 18:44:38.148483 2844 leader_election.cc:288] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:38.148577 2855 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:38.148751 2855 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [term 1 LEADER]: Becoming Leader. State: Replica: 5f937dac0e5a40969257882300039a23, State: Running, Role: LEADER
I20260806 18:44:38.149052 2855 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f937dac0e5a40969257882300039a23" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34817 } }
I20260806 18:44:38.149822 2844 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:38.150892 2858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f937dac0e5a40969257882300039a23. Latest consensus state: current_term: 1 leader_uuid: "5f937dac0e5a40969257882300039a23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f937dac0e5a40969257882300039a23" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34817 } } }
I20260806 18:44:38.151032 2858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:38.151485 2869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:38.151434 2856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5f937dac0e5a40969257882300039a23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5f937dac0e5a40969257882300039a23" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34817 } } }
I20260806 18:44:38.151592 2856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:38.154609 2869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:38.160352 2869 catalog_manager.cc:1347] Generated new cluster ID: a03b8f3c5d98483bb12104a68d27f422
I20260806 18:44:38.160425 2869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:38.184092 2869 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:38.185096 2869 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:38.189976 2869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5f937dac0e5a40969257882300039a23: Generated new TSK 0
I20260806 18:44:38.190735 2869 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:44:38.229264 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34817
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:38.344247 2886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:38.344519 2886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:38.344580 2886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:38.348747 2886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:38.348815 2886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:38.348902 2886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:44:38.354291 2886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34817
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:38.355346 2886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:38.356339 2886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:38.365686 2893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:38.365685 2895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:38.365865 2897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:38.365808 2886 server_base.cc:1034] running on GCE node
I20260806 18:44:38.366504 2886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:38.367129 2886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:38.368299 2886 hybrid_clock.cc:648] HybridClock initialized: now 1786041878368280 us; error 29 us; skew 500 ppm
Aug 06 18:44:38 dist-test-slave-fgb3 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:38.371330 2886 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:44:38.372159 2886 webserver.cc:460] Webserver started at http://127.1.245.65:33347/ using document root <none> and password file <none>
I20260806 18:44:38.372668 2886 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:38.372733 2886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:38.372915 2886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:38.374924 2886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "079fa9c82bdb404b9779b34be2522c50"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "98599701ed9b50c3df834ff64dd0dc6c"
server_key_iv: "122b9ee00732d03f669426a39dd6e8c5"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.375432 2886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "079fa9c82bdb404b9779b34be2522c50"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "98599701ed9b50c3df834ff64dd0dc6c"
server_key_iv: "122b9ee00732d03f669426a39dd6e8c5"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.379123 2886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:44:38.381551 2907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.382761 2886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:44:38.382884 2886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "079fa9c82bdb404b9779b34be2522c50"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "98599701ed9b50c3df834ff64dd0dc6c"
server_key_iv: "122b9ee00732d03f669426a39dd6e8c5"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.382969 2886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:38.405767 2886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:38.408710 2886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:38.408901 2886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:38.409507 2886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:38.410385 2886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:38.410435 2886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.410481 2886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:38.410508 2886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.419507 2886 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:33957
I20260806 18:44:38.419544 3069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:33957 every 8 connection(s)
I20260806 18:44:38.420423 2886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:44:38.426713 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2886
I20260806 18:44:38.426828 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:44:38.427064 2005 external_mini_cluster.cc:1428] Setting key b273bd2bc7b17ae9f5a965dc67faf646
Aug 06 18:44:38 dist-test-slave-fgb3 krb5kdc[2735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:38.435166 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.422392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:54651 (local address 127.1.245.126:34817)
0806 18:44:38.422520 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:38.422524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:38.423395 (+ 871us) server_negotiation.cc:407] Connection header received
0806 18:44:38.424712 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:38.424715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:38.424767 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:38.424846 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:38.426077 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.426725 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.428023 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.428294 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.428467 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.431259 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:38.431262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:38.431265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:38.431290 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:38.432674 (+ 1384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.433366 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.433370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.433371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.433463 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.433864 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.433866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.433867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.434056 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:38.434131 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:38.434898 (+ 767us) server_negotiation.cc:299] Negotiation successful
0806 18:44:38.435025 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:44:38.436400 3075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.422725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34817 (local address 127.1.245.65:54651)
0806 18:44:38.423199 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:38.423243 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:38.424483 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:38.425002 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:38.425008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:38.425409 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:38.425902 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.425916 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.426867 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.426871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.427906 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.427917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.428498 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.428502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.428706 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.429335 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:38.429357 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:38.431084 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:38.432814 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:38.432822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:38.432840 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:38.433240 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:38.433557 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:38.433560 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:38.433562 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:38.433777 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:38.434157 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:38.434164 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:38.434589 (+ 425us) client_negotiation.cc:769] Sending connection context
0806 18:44:38.434838 (+ 249us) client_negotiation.cc:240] Negotiation successful
0806 18:44:38.435235 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:44:38.437812 3070 heartbeater.cc:347] Connected to a master server at 127.1.245.126:34817
I20260806 18:44:38.438103 3070 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:38.438699 3070 heartbeater.cc:511] Master 127.1.245.126:34817 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:44:38.440433 2794 ts_manager.cc:194] Registered new tserver with Master: 079fa9c82bdb404b9779b34be2522c50 (127.1.245.65:33957)
I20260806 18:44:38.441907 2794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:54651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:44:38.457050 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34817
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:38.575078 3078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:38.575369 3078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:38.575417 3078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:38.579418 3078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:38.579484 3078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:38.579547 3078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:44:38.585037 3078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34817
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:38.586196 3078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:38.587304 3078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:38.597008 3088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:38.597033 3086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:38.597008 3085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:38.597252 3078 server_base.cc:1034] running on GCE node
I20260806 18:44:38.597811 3078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:38.598490 3078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:38.599673 3078 hybrid_clock.cc:648] HybridClock initialized: now 1786041878599648 us; error 37 us; skew 500 ppm
Aug 06 18:44:38 dist-test-slave-fgb3 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:38.602542 3078 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:44:38.603452 3078 webserver.cc:460] Webserver started at http://127.1.245.66:45117/ using document root <none> and password file <none>
I20260806 18:44:38.603998 3078 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:38.604056 3078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:38.604238 3078 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:38.606364 3078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "53ffac965dcf46be9d85c6b12d2595ad"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "4743104c9d1d1e500ff1f91734a729a4"
server_key_iv: "41c339e7d980c3472541b40d367bd92e"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.606930 3078 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "53ffac965dcf46be9d85c6b12d2595ad"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "4743104c9d1d1e500ff1f91734a729a4"
server_key_iv: "41c339e7d980c3472541b40d367bd92e"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.610785 3078 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:38.613291 3102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.614749 3078 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:44:38.614868 3078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "53ffac965dcf46be9d85c6b12d2595ad"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "4743104c9d1d1e500ff1f91734a729a4"
server_key_iv: "41c339e7d980c3472541b40d367bd92e"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.614979 3078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:38.647450 3078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:38.650959 3078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:38.651165 3078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:38.651810 3078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:38.652671 3078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:38.652721 3078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.652774 3078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:38.652812 3078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.661482 3078 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:39189
I20260806 18:44:38.661511 3257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:39189 every 8 connection(s)
I20260806 18:44:38.662429 3078 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:44:38.664364 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 3078
I20260806 18:44:38.664464 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:44:38.664697 2005 external_mini_cluster.cc:1428] Setting key 6d693a66b737347a25dbd33d1e8d038e
Aug 06 18:44:38 dist-test-slave-fgb3 krb5kdc[2735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:44:38.676375 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.664266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:38019 (local address 127.1.245.126:34817)
0806 18:44:38.664423 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:38.664429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:38.665868 (+ 1439us) server_negotiation.cc:407] Connection header received
0806 18:44:38.667042 (+ 1174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:38.667046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:38.667090 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:38.667163 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:38.668541 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.669062 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.670163 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.670427 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.670517 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.673070 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:38.673073 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:38.673076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:38.673100 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:38.674491 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.674974 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.674976 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.674977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.675048 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.675445 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.675447 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.675448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.675621 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:38.675710 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:38.676133 (+ 423us) server_negotiation.cc:299] Negotiation successful
0806 18:44:38.676252 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
I20260806 18:44:38.677405 3260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.664741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34817 (local address 127.1.245.66:38019)
0806 18:44:38.665237 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:38.665293 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:38.666790 (+ 1497us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:38.667361 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:38.667369 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:38.667839 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:38.668395 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.668409 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.669176 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.669180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.670049 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.670058 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.670564 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.670568 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.670732 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.671354 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:38.671370 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:38.672898 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:38.674598 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:38.674603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:38.674615 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:38.674881 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:38.675150 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:38.675153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:38.675154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:38.675365 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:38.675696 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:38.675701 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:38.675942 (+ 241us) client_negotiation.cc:769] Sending connection context
0806 18:44:38.676099 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 18:44:38.676431 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":121,"threads_started":1}
I20260806 18:44:38.678606 3258 heartbeater.cc:347] Connected to a master server at 127.1.245.126:34817
I20260806 18:44:38.678862 3258 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:38.679350 3258 heartbeater.cc:511] Master 127.1.245.126:34817 requested a full tablet report, sending...
I20260806 18:44:38.680409 2794 ts_manager.cc:194] Registered new tserver with Master: 53ffac965dcf46be9d85c6b12d2595ad (127.1.245.66:39189)
I20260806 18:44:38.681180 2794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:38019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:44:38.693071 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34817
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:38.803582 3264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:38.803843 3264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:38.803886 3264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:38.807883 3264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:38.807942 3264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:38.808000 3264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:44:38.813125 3264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34817
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:38.814224 3264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:38.815282 3264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:38.824906 3273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:38.825002 3264 server_base.cc:1034] running on GCE node
W20260806 18:44:38.824946 3276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:38.824920 3272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:38.825551 3264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:38.826202 3264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:38.827365 3264 hybrid_clock.cc:648] HybridClock initialized: now 1786041878827347 us; error 34 us; skew 500 ppm
Aug 06 18:44:38 dist-test-slave-fgb3 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:38.830013 3264 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:44:38.830960 3264 webserver.cc:460] Webserver started at http://127.1.245.67:43353/ using document root <none> and password file <none>
I20260806 18:44:38.831462 3264 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:38.831519 3264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:38.831676 3264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:38.833611 3264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "f1a207d1dbed42ff8378e361d7f0d487"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "0e350a06a8727bf16b2d65fd7c0e3ff9"
server_key_iv: "e436d48b8dfad6f401085f34068d24ee"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.834137 3264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "f1a207d1dbed42ff8378e361d7f0d487"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "0e350a06a8727bf16b2d65fd7c0e3ff9"
server_key_iv: "e436d48b8dfad6f401085f34068d24ee"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.837769 3264 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:44:38.840274 3284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.841485 3264 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:38.841583 3264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "f1a207d1dbed42ff8378e361d7f0d487"
format_stamp: "Formatted at 2026-08-06 18:44:38 on dist-test-slave-fgb3"
server_key: "0e350a06a8727bf16b2d65fd7c0e3ff9"
server_key_iv: "e436d48b8dfad6f401085f34068d24ee"
server_key_version: "encryptionkey@0"
I20260806 18:44:38.841687 3264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:38.858026 3264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:38.860994 3264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:38.861169 3264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:38.861757 3264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:38.862620 3264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:38.862669 3264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.862723 3264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:38.862751 3264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:38.871441 3264 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:40033
I20260806 18:44:38.871480 3447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:40033 every 8 connection(s)
I20260806 18:44:38.872334 3264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:44:38.879902 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 3264
I20260806 18:44:38.880002 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:44:38.880286 2005 external_mini_cluster.cc:1428] Setting key 241f202c825851db41074fd7562415d3
Aug 06 18:44:38 dist-test-slave-fgb3 krb5kdc[2735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041878, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:38.885529 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.874113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:56949 (local address 127.1.245.126:34817)
0806 18:44:38.874266 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:38.874269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:38.875112 (+ 843us) server_negotiation.cc:407] Connection header received
0806 18:44:38.876445 (+ 1333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:38.876449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:38.876507 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:38.876585 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:38.877744 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.878281 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.879407 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.879619 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.879727 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.882464 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:38.882468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:38.882471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:38.882500 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:38.883620 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.884084 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.884088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.884090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.884169 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.884492 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.884496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.884497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.884680 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:38.884762 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:38.885247 (+ 485us) server_negotiation.cc:299] Negotiation successful
0806 18:44:38.885370 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:44:38.886353 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.874521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34817 (local address 127.1.245.67:56949)
0806 18:44:38.874962 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:38.875006 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:38.876230 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:38.876762 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:38.876769 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:38.877118 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:38.877598 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.877612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.878418 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.878421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.879285 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.879295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.879767 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.879772 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.879941 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.880565 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:38.880582 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:38.882249 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:38.883725 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:38.883731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:38.883746 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:38.883993 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:38.884250 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:38.884253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:38.884255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:38.884386 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:38.884775 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:38.884782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:38.885022 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 18:44:38.885194 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:44:38.885521 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":115,"threads_started":1}
I20260806 18:44:38.887511 3448 heartbeater.cc:347] Connected to a master server at 127.1.245.126:34817
I20260806 18:44:38.887732 3448 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:38.888217 3448 heartbeater.cc:511] Master 127.1.245.126:34817 requested a full tablet report, sending...
I20260806 18:44:38.889323 2794 ts_manager.cc:194] Registered new tserver with Master: f1a207d1dbed42ff8378e361d7f0d487 (127.1.245.67:40033)
I20260806 18:44:38.889879 2794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:56949
I20260806 18:44:38.894443 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:44:38.904628 2852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.898017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.1.245.126:34817)
0806 18:44:38.898139 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:38.898143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:38.898377 (+ 234us) server_negotiation.cc:407] Connection header received
0806 18:44:38.898498 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:38.898500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:38.898541 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:38.898615 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:38.899321 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.899811 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.900557 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.900750 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.900814 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.902050 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:38.902053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:38.902055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:38.902077 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:38.903262 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.903635 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.903638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.903639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.903700 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:38.903908 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:38.903909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:38.903910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:38.904111 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:38.904205 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:38.904421 (+ 216us) server_negotiation.cc:299] Negotiation successful
0806 18:44:38.904511 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:44:38.909205 2794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:44:38.912215 2794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:38.926707 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.920549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37896 (local address 127.1.245.66:39189)
0806 18:44:38.921770 (+ 1221us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:38.921782 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:38.921804 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:44:38.921881 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:38.921886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:38.922081 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:38.922271 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:38.922591 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.923768 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.925315 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.926090 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.926192 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.926360 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:38.926458 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 18:44:38.926547 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1124,"thread_start_us":107,"threads_started":1}
I20260806 18:44:38.926702 3463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.920409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:39189 (local address 127.0.0.1:37896)
0806 18:44:38.921041 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:38.921091 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:38.921220 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:38.922262 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:38.922268 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:38.922291 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:38.922470 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.922478 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.923891 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.923894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.925195 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.925206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.926211 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.926215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.926302 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.926348 (+ 46us) client_negotiation.cc:769] Sending connection context
0806 18:44:38.926436 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 18:44:38.926547 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":56,"thread_start_us":90,"threads_started":1}
I20260806 18:44:38.926762 3469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.921207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.1.245.67:40033)
0806 18:44:38.921522 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:38.921533 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:38.921697 (+ 164us) server_negotiation.cc:407] Connection header received
0806 18:44:38.921823 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:38.921829 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:38.922053 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:38.922203 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:38.922918 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.923995 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.925058 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.925796 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.925900 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.926049 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:38.926537 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 18:44:38.926619 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":126,"threads_started":1}
I20260806 18:44:38.927026 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.921216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47146 (local address 127.1.245.65:33957)
0806 18:44:38.921565 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:38.921579 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:38.921988 (+ 409us) server_negotiation.cc:407] Connection header received
0806 18:44:38.922185 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:38.922191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:38.922380 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:38.922486 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:38.922873 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.923659 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.925336 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.926121 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.926215 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.926385 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:38.926770 (+ 385us) server_negotiation.cc:299] Negotiation successful
0806 18:44:38.926866 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":121,"threads_started":1}
I20260806 18:44:38.927210 3464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.920791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:33957 (local address 127.0.0.1:47146)
0806 18:44:38.921900 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:38.921912 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:38.922048 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:38.922652 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:38.922655 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:38.922673 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:38.922766 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.922771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.923781 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.923784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.925195 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.925206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.926440 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.926444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.926521 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.926540 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:44:38.926600 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:44:38.926654 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1053,"mutex_wait_us":40,"thread_start_us":91,"threads_started":1}
I20260806 18:44:38.927371 3468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:38.920528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:40033 (local address 127.0.0.1:33924)
0806 18:44:38.921588 (+ 1060us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:38.921601 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:38.921703 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:38.922393 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:38.922395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:38.922409 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:38.922482 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.922487 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.924117 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.924120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.924931 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:38.924940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:38.926118 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:38.926122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:38.926229 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:38.926269 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:44:38.926409 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 18:44:38.927244 (+ 835us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":715,"thread_start_us":93,"threads_started":1}
I20260806 18:44:38.929787 3167 tablet_service.cc:1467] Processing CreateTablet for tablet b29f8ab7ff974df5b95ccce4b4e8a4a3 (DEFAULT_TABLE table=test-table [id=663df42713f340deb1d6d855323e60ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:38.929931 3351 tablet_service.cc:1467] Processing CreateTablet for tablet b29f8ab7ff974df5b95ccce4b4e8a4a3 (DEFAULT_TABLE table=test-table [id=663df42713f340deb1d6d855323e60ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:38.929921 2973 tablet_service.cc:1467] Processing CreateTablet for tablet b29f8ab7ff974df5b95ccce4b4e8a4a3 (DEFAULT_TABLE table=test-table [id=663df42713f340deb1d6d855323e60ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:38.930886 3167 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b29f8ab7ff974df5b95ccce4b4e8a4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:38.930938 2973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b29f8ab7ff974df5b95ccce4b4e8a4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:38.930944 3351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b29f8ab7ff974df5b95ccce4b4e8a4a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:38.937063 3471 tablet_bootstrap.cc:492] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad: Bootstrap starting.
I20260806 18:44:38.938606 3473 tablet_bootstrap.cc:492] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487: Bootstrap starting.
I20260806 18:44:38.938753 3472 tablet_bootstrap.cc:492] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50: Bootstrap starting.
I20260806 18:44:38.940362 3471 tablet_bootstrap.cc:654] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:38.940783 3472 tablet_bootstrap.cc:654] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:38.940872 3473 tablet_bootstrap.cc:654] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:38.941285 3471 log.cc:824] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:38.941550 3473 log.cc:824] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:38.941570 3472 log.cc:824] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:38.943316 3473 tablet_bootstrap.cc:492] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487: No bootstrap required, opened a new log
I20260806 18:44:38.943316 3472 tablet_bootstrap.cc:492] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50: No bootstrap required, opened a new log
I20260806 18:44:38.943491 3472 ts_tablet_manager.cc:1397] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:44:38.943490 3473 ts_tablet_manager.cc:1397] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 18:44:38.946444 3471 tablet_bootstrap.cc:492] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad: No bootstrap required, opened a new log
I20260806 18:44:38.946638 3471 ts_tablet_manager.cc:1397] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260806 18:44:38.946816 3473 raft_consensus.cc:348] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:38.947065 3473 raft_consensus.cc:374] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:38.947125 3473 raft_consensus.cc:729] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1a207d1dbed42ff8378e361d7f0d487, State: Initialized, Role: FOLLOWER
I20260806 18:44:38.947544 3473 consensus_queue.cc:260] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:38.947692 3472 raft_consensus.cc:348] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:38.947945 3472 raft_consensus.cc:374] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:38.948001 3472 raft_consensus.cc:729] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 079fa9c82bdb404b9779b34be2522c50, State: Initialized, Role: FOLLOWER
I20260806 18:44:38.948272 3448 heartbeater.cc:503] Master 127.1.245.126:34817 was elected leader, sending a full tablet report...
I20260806 18:44:38.948526 3473 ts_tablet_manager.cc:1428] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260806 18:44:38.948454 3472 consensus_queue.cc:260] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:38.949373 3472 ts_tablet_manager.cc:1428] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260806 18:44:38.949716 3070 heartbeater.cc:503] Master 127.1.245.126:34817 was elected leader, sending a full tablet report...
I20260806 18:44:38.950340 3471 raft_consensus.cc:348] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:38.950578 3471 raft_consensus.cc:374] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:38.950642 3471 raft_consensus.cc:729] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53ffac965dcf46be9d85c6b12d2595ad, State: Initialized, Role: FOLLOWER
I20260806 18:44:38.951001 3471 consensus_queue.cc:260] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:38.951658 3258 heartbeater.cc:503] Master 127.1.245.126:34817 was elected leader, sending a full tablet report...
I20260806 18:44:38.951912 3471 ts_tablet_manager.cc:1428] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 18:44:39.123261 3449 tablet.cc:2369] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:44:39.163533 3259 tablet.cc:2369] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:44:39.171886 3073 tablet.cc:2369] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:44:39.362488 3479 raft_consensus.cc:482] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:39.362715 3479 raft_consensus.cc:504] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:39.363929 3479 leader_election.cc:288] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1a207d1dbed42ff8378e361d7f0d487 (127.1.245.67:40033), 079fa9c82bdb404b9779b34be2522c50 (127.1.245.65:33957)
I20260806 18:44:39.367727 3478 raft_consensus.cc:482] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:39.367940 3478 raft_consensus.cc:504] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:39.368943 3469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.364262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:49427 (local address 127.1.245.67:40033)
0806 18:44:39.364443 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.364448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.364671 (+ 223us) server_negotiation.cc:407] Connection header received
0806 18:44:39.364812 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.364816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.364870 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.364953 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:39.365315 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.366098 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.367912 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.368324 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.368437 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.368495 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.368716 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.368808 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 18:44:39.369077 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.364195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:33957 (local address 127.1.245.66:41427)
0806 18:44:39.364591 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:39.364604 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:39.364711 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:39.364969 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:39.364972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:39.365010 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:39.365189 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.365195 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.366216 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.366220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.367724 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.367737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.368680 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.368684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.368778 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.368801 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:44:39.368870 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:44:39.368939 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":78,"spinlock_wait_cycles":45696,"thread_start_us":74,"threads_started":1}
I20260806 18:44:39.369110 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.364384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:41427 (local address 127.1.245.65:33957)
0806 18:44:39.364511 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.364530 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.364666 (+ 136us) server_negotiation.cc:407] Connection header received
0806 18:44:39.364812 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.364816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.364870 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.364953 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:39.365297 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.366098 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.367893 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.368534 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.368669 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.368727 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.368900 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.368982 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:44:39.369340 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.364175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:40033 (local address 127.1.245.66:49427)
0806 18:44:39.364560 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:39.364574 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:39.364710 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:39.364969 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:39.364972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:39.365011 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:39.365177 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.365185 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.366216 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.366220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.367724 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.367736 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.368480 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.368484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.368579 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.368600 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:44:39.369133 (+ 533us) client_negotiation.cc:240] Negotiation successful
0806 18:44:39.369209 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":157,"spinlock_wait_cycles":7680,"thread_start_us":169,"threads_started":1}
I20260806 18:44:39.369321 3478 leader_election.cc:288] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ffac965dcf46be9d85c6b12d2595ad (127.1.245.66:39189), f1a207d1dbed42ff8378e361d7f0d487 (127.1.245.67:40033)
I20260806 18:44:39.369990 3384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f8ab7ff974df5b95ccce4b4e8a4a3" candidate_uuid: "53ffac965dcf46be9d85c6b12d2595ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a207d1dbed42ff8378e361d7f0d487" is_pre_election: true
I20260806 18:44:39.370195 3477 raft_consensus.cc:482] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:39.370369 3477 raft_consensus.cc:504] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:39.370560 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f8ab7ff974df5b95ccce4b4e8a4a3" candidate_uuid: "53ffac965dcf46be9d85c6b12d2595ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "079fa9c82bdb404b9779b34be2522c50" is_pre_election: true
I20260806 18:44:39.370771 3004 raft_consensus.cc:2454] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ffac965dcf46be9d85c6b12d2595ad in term 0.
I20260806 18:44:39.371075 3384 raft_consensus.cc:2454] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 53ffac965dcf46be9d85c6b12d2595ad in term 0.
I20260806 18:44:39.371276 3103 leader_election.cc:302] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 079fa9c82bdb404b9779b34be2522c50, 53ffac965dcf46be9d85c6b12d2595ad; no voters:
I20260806 18:44:39.371932 3479 raft_consensus.cc:2781] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:39.372033 3479 raft_consensus.cc:482] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:39.372079 3479 raft_consensus.cc:3037] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:39.372197 3477 leader_election.cc:288] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53ffac965dcf46be9d85c6b12d2595ad (127.1.245.66:39189), 079fa9c82bdb404b9779b34be2522c50 (127.1.245.65:33957)
I20260806 18:44:39.373500 3479 raft_consensus.cc:504] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:39.373934 3479 leader_election.cc:288] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [CANDIDATE]: Term 1 election: Requested vote from peers f1a207d1dbed42ff8378e361d7f0d487 (127.1.245.67:40033), 079fa9c82bdb404b9779b34be2522c50 (127.1.245.65:33957)
I20260806 18:44:39.374238 3384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f8ab7ff974df5b95ccce4b4e8a4a3" candidate_uuid: "53ffac965dcf46be9d85c6b12d2595ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a207d1dbed42ff8378e361d7f0d487"
I20260806 18:44:39.374399 3384 raft_consensus.cc:3037] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:39.374711 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f8ab7ff974df5b95ccce4b4e8a4a3" candidate_uuid: "53ffac965dcf46be9d85c6b12d2595ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "079fa9c82bdb404b9779b34be2522c50"
I20260806 18:44:39.374837 3004 raft_consensus.cc:3037] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:39.374945 3469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.369945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:55533 (local address 127.1.245.67:40033)
0806 18:44:39.370133 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.370137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.370605 (+ 468us) server_negotiation.cc:407] Connection header received
0806 18:44:39.370764 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.370767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.370819 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.370893 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:39.371550 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.372181 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.373750 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.374229 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.374372 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.374419 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.374762 (+ 343us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.374842 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128}
I20260806 18:44:39.375227 3483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.369575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:40033 (local address 127.1.245.65:55533)
0806 18:44:39.370492 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:39.370532 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:39.370650 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:39.371058 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:39.371061 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:39.371080 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:39.371309 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.371321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.372386 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.372390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.373629 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.373641 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.374429 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.374433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.374541 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.374565 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:44:39.374635 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:44:39.374702 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":141,"threads_started":1}
I20260806 18:44:39.375778 3384 raft_consensus.cc:2454] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ffac965dcf46be9d85c6b12d2595ad in term 1.
I20260806 18:44:39.375902 3004 raft_consensus.cc:2454] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 53ffac965dcf46be9d85c6b12d2595ad in term 1.
I20260806 18:44:39.376238 3103 leader_election.cc:302] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 079fa9c82bdb404b9779b34be2522c50, 53ffac965dcf46be9d85c6b12d2595ad; no voters:
I20260806 18:44:39.376420 3479 raft_consensus.cc:2781] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:39.376545 3384 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f8ab7ff974df5b95ccce4b4e8a4a3" candidate_uuid: "079fa9c82bdb404b9779b34be2522c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1a207d1dbed42ff8378e361d7f0d487" is_pre_election: true
I20260806 18:44:39.376605 3479 raft_consensus.cc:686] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [term 1 LEADER]: Becoming Leader. State: Replica: 53ffac965dcf46be9d85c6b12d2595ad, State: Running, Role: LEADER
I20260806 18:44:39.376735 3384 raft_consensus.cc:2379] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 079fa9c82bdb404b9779b34be2522c50 in current term 1: Already voted for candidate 53ffac965dcf46be9d85c6b12d2595ad in this term.
I20260806 18:44:39.376920 3479 consensus_queue.cc:237] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } }
I20260806 18:44:39.378372 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.371632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:33957 (local address 127.1.245.67:42255)
0806 18:44:39.374100 (+ 2468us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:39.374115 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:39.374203 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:39.374911 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:39.374915 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:39.374932 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:39.375032 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.375039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.375976 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.375979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.377105 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.377116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.377884 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.377889 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.378017 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.378044 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:44:39.378127 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:44:39.378199 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2397}
I20260806 18:44:39.379009 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.371850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:42255 (local address 127.1.245.65:33957)
0806 18:44:39.372165 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.372170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.374632 (+ 2462us) server_negotiation.cc:407] Connection header received
0806 18:44:39.374676 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.374679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.374734 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.374810 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:39.375162 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.375848 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.377248 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.377771 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.377895 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.378000 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.378824 (+ 824us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.378894 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":247}
I20260806 18:44:39.379362 3004 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f8ab7ff974df5b95ccce4b4e8a4a3" candidate_uuid: "f1a207d1dbed42ff8378e361d7f0d487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "079fa9c82bdb404b9779b34be2522c50" is_pre_election: true
I20260806 18:44:39.379536 3004 raft_consensus.cc:2379] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1a207d1dbed42ff8378e361d7f0d487 in current term 1: Already voted for candidate 53ffac965dcf46be9d85c6b12d2595ad in this term.
I20260806 18:44:39.380750 3465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.369643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:40835 (local address 127.1.245.66:39189)
0806 18:44:39.370054 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.370058 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.370987 (+ 929us) server_negotiation.cc:407] Connection header received
0806 18:44:39.376465 (+ 5478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.376468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.376522 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.376578 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:39.377018 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.377633 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.379084 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.379640 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.379757 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.379842 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.380539 (+ 697us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.380615 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338}
I20260806 18:44:39.381156 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.369592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:39189 (local address 127.1.245.65:40835)
0806 18:44:39.370905 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:39.370919 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:39.376001 (+ 5082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:39.376739 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:39.376742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:39.376763 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:39.376879 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.376887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.377796 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.377799 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.378939 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.378948 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.380274 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.380278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.380384 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.380410 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:44:39.380939 (+ 529us) client_negotiation.cc:240] Negotiation successful
0806 18:44:39.381015 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1249,"thread_start_us":128,"threads_started":1}
I20260806 18:44:39.380833 2793 catalog_manager.cc:5495] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 53ffac965dcf46be9d85c6b12d2595ad reported cstate change: term changed from 0 to 1, leader changed from <none> to 53ffac965dcf46be9d85c6b12d2595ad (127.1.245.66). New cstate: current_term: 1 leader_uuid: "53ffac965dcf46be9d85c6b12d2595ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53ffac965dcf46be9d85c6b12d2595ad" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39189 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1a207d1dbed42ff8378e361d7f0d487" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 40033 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "079fa9c82bdb404b9779b34be2522c50" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33957 } health_report { overall_health: UNKNOWN } } }
I20260806 18:44:39.381703 3194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f8ab7ff974df5b95ccce4b4e8a4a3" candidate_uuid: "079fa9c82bdb404b9779b34be2522c50" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ffac965dcf46be9d85c6b12d2595ad" is_pre_election: true
I20260806 18:44:39.382336 2909 leader_election.cc:302] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 079fa9c82bdb404b9779b34be2522c50; no voters: 53ffac965dcf46be9d85c6b12d2595ad, f1a207d1dbed42ff8378e361d7f0d487
I20260806 18:44:39.382688 3478 raft_consensus.cc:2726] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P 079fa9c82bdb404b9779b34be2522c50 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 18:44:39.383263 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.371632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:39189 (local address 127.1.245.67:52161)
0806 18:44:39.378527 (+ 6895us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:39.378541 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:39.378622 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:39.380030 (+ 1408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:39.380033 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:39.380051 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:39.380125 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.380131 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.381066 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.381069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.382021 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:39.382031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.382903 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.382907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:39.382992 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.383019 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:44:39.383076 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 18:44:39.383147 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6840,"mutex_wait_us":252,"thread_start_us":88,"threads_started":1}
I20260806 18:44:39.383826 3484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.371749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:52161 (local address 127.1.245.66:39189)
0806 18:44:39.372235 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.372243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.379803 (+ 7560us) server_negotiation.cc:407] Connection header received
0806 18:44:39.379848 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.379851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.379905 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.380226 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:39.380325 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.380949 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.382153 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.382750 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.383563 (+ 813us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.383602 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.383633 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.383688 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":90,"threads_started":1}
I20260806 18:44:39.384222 3194 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b29f8ab7ff974df5b95ccce4b4e8a4a3" candidate_uuid: "f1a207d1dbed42ff8378e361d7f0d487" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53ffac965dcf46be9d85c6b12d2595ad" is_pre_election: true
I20260806 18:44:39.384677 3288 leader_election.cc:302] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1a207d1dbed42ff8378e361d7f0d487; no voters: 079fa9c82bdb404b9779b34be2522c50, 53ffac965dcf46be9d85c6b12d2595ad
I20260806 18:44:39.384967 3477 raft_consensus.cc:2726] T b29f8ab7ff974df5b95ccce4b4e8a4a3 P f1a207d1dbed42ff8378e361d7f0d487 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
Aug 06 18:44:39 dist-test-slave-fgb3 krb5kdc[2735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:44:39.497056 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.484782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47150 (local address 127.1.245.65:33957)
0806 18:44:39.484938 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.484943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.485026 (+ 83us) server_negotiation.cc:407] Connection header received
0806 18:44:39.485163 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.485166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.485218 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.485286 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:39.485981 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.486549 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.487404 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.487620 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.487710 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.493663 (+ 5953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:39.493678 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:39.493697 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:39.493724 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:39.495411 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:39.495849 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:39.495854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:39.495856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:39.495936 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:39.496182 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:39.496186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:39.496188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:39.496470 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:39.496647 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.496832 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.496931 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 06 18:44:39 dist-test-slave-fgb3 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041879, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:44:39 dist-test-slave-fgb3 krb5kdc[2735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041879, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:44:39.523911 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.515683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47152 (local address 127.1.245.65:33957)
0806 18:44:39.515844 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.515850 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.515949 (+ 99us) server_negotiation.cc:407] Connection header received
0806 18:44:39.516109 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.516112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.516158 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.516223 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:39.516919 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.517443 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.518298 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.518570 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.518662 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.520991 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:39.520995 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:39.520997 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:39.521024 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:39.522279 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:39.522741 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:39.522745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:39.522747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:39.522824 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:39.523067 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:39.523071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:39.523073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:39.523239 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:39.523367 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.523639 (+ 272us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.523753 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 18:44:39 dist-test-slave-fgb3 krb5kdc[2735](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:44:39 dist-test-slave-fgb3 krb5kdc[2735](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:44:39.549170 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:39.541070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47164 (local address 127.1.245.65:33957)
0806 18:44:39.541211 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:39.541216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:39.541296 (+ 80us) server_negotiation.cc:407] Connection header received
0806 18:44:39.541473 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:39.541476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:39.541521 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:39.541591 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:39.542281 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.542889 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.543783 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:39.544006 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:39.544094 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:39.546398 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:39.546402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:39.546405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:39.546436 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:39.547656 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:39.548030 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:39.548035 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:39.548037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:39.548118 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:39.548357 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:39.548360 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:39.548362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:39.548532 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:39.548666 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:39.548901 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 18:44:39.549010 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:44:39.550854 2942 tablet_service.cc:2908] Rejecting scan request for tablet b29f8ab7ff974df5b95ccce4b4e8a4a3: Uninitialized: clean time has not yet been initialized
I20260806 18:44:39.552838 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2886
I20260806 18:44:39.565143 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 3078
I20260806 18:44:39.576143 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 3264
I20260806 18:44:39.586619 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2753
2026-08-06T18:44:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2904 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:39 dist-test-slave-fgb3 krb5kdc[3509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:39 dist-test-slave-fgb3 krb5kdc[3509](info): set up 2 sockets
Aug 06 18:44:39 dist-test-slave-fgb3 krb5kdc[3509](info): commencing operation
krb5kdc: starting...
W20260806 18:44:40.705554 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:40 dist-test-slave-fgb3 krb5kdc[3509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:40.825400 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33021
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:33021
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:40.937570 3528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:40.937846 3528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:40.937903 3528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:40.941866 3528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:40.941924 3528 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:44:40.941946 3528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:40.941968 3528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:40.941990 3528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:40.942018 3528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:40.947227 3528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:33021
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:40.948230 3528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:40.949203 3528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:40.957556 3535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:40.957563 3537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:40.957564 3534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:40.957613 3528 server_base.cc:1034] running on GCE node
I20260806 18:44:40.958371 3528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:40.959452 3528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:40.960639 3528 hybrid_clock.cc:648] HybridClock initialized: now 1786041880960614 us; error 35 us; skew 500 ppm
Aug 06 18:44:40 dist-test-slave-fgb3 krb5kdc[3509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041880, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:40.963420 3528 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:40.964388 3528 webserver.cc:460] Webserver started at http://127.1.245.126:44829/ using document root <none> and password file <none>
I20260806 18:44:40.964921 3528 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:40.964982 3528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:40.965183 3528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:40.967391 3528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "a6f54db4b7ec4f249663939007cad5ab"
format_stamp: "Formatted at 2026-08-06 18:44:40 on dist-test-slave-fgb3"
server_key: "4b371ec3920603245b40f71984562922"
server_key_iv: "bfde4d5ea8909d2ff68122d77a9ebf63"
server_key_version: "encryptionkey@0"
I20260806 18:44:40.967932 3528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "a6f54db4b7ec4f249663939007cad5ab"
format_stamp: "Formatted at 2026-08-06 18:44:40 on dist-test-slave-fgb3"
server_key: "4b371ec3920603245b40f71984562922"
server_key_iv: "bfde4d5ea8909d2ff68122d77a9ebf63"
server_key_version: "encryptionkey@0"
I20260806 18:44:40.971522 3528 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:44:40.973841 3547 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:40.975076 3528 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:40.975185 3528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "a6f54db4b7ec4f249663939007cad5ab"
format_stamp: "Formatted at 2026-08-06 18:44:40 on dist-test-slave-fgb3"
server_key: "4b371ec3920603245b40f71984562922"
server_key_iv: "bfde4d5ea8909d2ff68122d77a9ebf63"
server_key_version: "encryptionkey@0"
I20260806 18:44:40.975291 3528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:40.985049 3528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:40.988291 3528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:40.988482 3528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:40.997545 3528 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:33021
I20260806 18:44:40.997568 3636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:33021 every 8 connection(s)
I20260806 18:44:40.998647 3528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:41.001394 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 3528
I20260806 18:44:41.001511 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:41.001653 3638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:41.001806 2005 external_mini_cluster.cc:1428] Setting key 611d34e9b82c290e716add33ae7c0308
I20260806 18:44:41.008708 3638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab: Bootstrap starting.
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:41.011442 3638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:41.012245 3638 log.cc:824] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:41.014514 3638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab: No bootstrap required, opened a new log
I20260806 18:44:41.017030 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.003297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.1.245.126:33021)
0806 18:44:41.003704 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.003721 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.003758 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:44:41.004386 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.004409 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.004745 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.005016 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:41.005895 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.006943 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.008072 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.008559 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.008688 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.011350 (+ 2662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:41.011356 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:41.011370 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:41.011420 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:41.013971 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.014431 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.014437 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.014443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.014604 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.014854 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.014858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.014859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.015355 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:41.015557 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.015754 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.016050 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":119,"threads_started":1}
I20260806 18:44:41.018457 3638 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f54db4b7ec4f249663939007cad5ab" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33021 } }
I20260806 18:44:41.018687 3638 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:41.018734 3638 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6f54db4b7ec4f249663939007cad5ab, State: Initialized, Role: FOLLOWER
I20260806 18:44:41.019176 3638 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f54db4b7ec4f249663939007cad5ab" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33021 } }
I20260806 18:44:41.019325 3638 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:41.019429 3638 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:41.019541 3638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:41.020516 3638 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f54db4b7ec4f249663939007cad5ab" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33021 } }
I20260806 18:44:41.020836 3638 leader_election.cc:302] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6f54db4b7ec4f249663939007cad5ab; no voters:
I20260806 18:44:41.021140 3638 leader_election.cc:288] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:41.021291 3645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:41.021507 3645 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [term 1 LEADER]: Becoming Leader. State: Replica: a6f54db4b7ec4f249663939007cad5ab, State: Running, Role: LEADER
I20260806 18:44:41.021847 3645 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f54db4b7ec4f249663939007cad5ab" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33021 } }
I20260806 18:44:41.022411 3638 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:41.023492 3649 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6f54db4b7ec4f249663939007cad5ab. Latest consensus state: current_term: 1 leader_uuid: "a6f54db4b7ec4f249663939007cad5ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f54db4b7ec4f249663939007cad5ab" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33021 } } }
I20260806 18:44:41.023808 3648 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6f54db4b7ec4f249663939007cad5ab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6f54db4b7ec4f249663939007cad5ab" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33021 } } }
I20260806 18:44:41.023921 3649 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:41.023955 3648 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab [sys.catalog]: This master's current role is: LEADER
W20260806 18:44:41.027375 3665 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:41.027454 3665 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:41.027572 3660 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:41.028278 3660 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:41.034379 3660 catalog_manager.cc:1347] Generated new cluster ID: 717f1b6b91ab4acbbf44a9e3181294c8
I20260806 18:44:41.034449 3660 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:41.045332 3660 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:41.046698 3660 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:41.051131 3660 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab: Generated new TSK 0
I20260806 18:44:41.051934 3660 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:44:41.088831 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33021
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:41.203852 3676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:41.204138 3676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:41.204206 3676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:41.208334 3676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:41.208420 3676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:41.208477 3676 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:44:41.208534 3676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:44:41.214068 3676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:33021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:41.215274 3676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:41.216315 3676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:44:41.226292 3676 server_base.cc:1034] running on GCE node
W20260806 18:44:41.226076 3694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:41.226076 3685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:41.226058 3688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:41.226874 3676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:41.227491 3676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:41.228706 3676 hybrid_clock.cc:648] HybridClock initialized: now 1786041881228681 us; error 34 us; skew 500 ppm
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:41.231642 3676 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:44:41.232534 3676 webserver.cc:460] Webserver started at http://127.1.245.65:40055/ using document root <none> and password file <none>
I20260806 18:44:41.233091 3676 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:41.233160 3676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:41.233353 3676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:41.235505 3676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "57de79e6b48b4a8bbb74d94e70c87552"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "dda279bfa00a17338688c700360c463d"
server_key_iv: "1fcf1223e83461f6d0018d0cc10da99d"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.236048 3676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "57de79e6b48b4a8bbb74d94e70c87552"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "dda279bfa00a17338688c700360c463d"
server_key_iv: "1fcf1223e83461f6d0018d0cc10da99d"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.239890 3676 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:44:41.242208 3704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:41.243537 3676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:41.243660 3676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "57de79e6b48b4a8bbb74d94e70c87552"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "dda279bfa00a17338688c700360c463d"
server_key_iv: "1fcf1223e83461f6d0018d0cc10da99d"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.243767 3676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:41.264750 3676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:41.267797 3676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:41.267997 3676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:41.269523 3676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:41.269583 3676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:41.269644 3676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:41.269677 3676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:41.280988 3676 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:37827
I20260806 18:44:41.281062 3873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:37827 every 8 connection(s)
I20260806 18:44:41.281931 3676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:44:41.283192 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.270431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:50253 (local address 127.1.245.126:33021)
0806 18:44:41.270555 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.270559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.271653 (+ 1094us) server_negotiation.cc:407] Connection header received
0806 18:44:41.272709 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.272713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.272775 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.272848 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:41.274541 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.275122 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.276308 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.276605 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.276697 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.279517 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:41.279521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:41.279524 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:41.279552 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:41.281107 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.281597 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.281601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.281603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.281682 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.282005 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.282009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.282011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.282232 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:41.282341 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.282901 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.283029 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:44:41.283941 3717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.271004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33021 (local address 127.1.245.65:50253)
0806 18:44:41.271510 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.271551 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.272458 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.273270 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.273279 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.273742 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:41.274393 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.274413 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.275247 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.275251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.276191 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.276201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.276779 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.276784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.276951 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.277550 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:41.277572 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:41.279358 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:41.281233 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.281238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.281250 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.281490 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.281761 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.281764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.281766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.281906 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.282352 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:41.282358 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:41.282650 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.282831 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.283138 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":156,"threads_started":1}
I20260806 18:44:41.285118 3875 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33021
I20260806 18:44:41.285254 3875 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:41.285735 3875 heartbeater.cc:511] Master 127.1.245.126:33021 requested a full tablet report, sending...
I20260806 18:44:41.286074 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 3676
I20260806 18:44:41.286186 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:44:41.286518 2005 external_mini_cluster.cc:1428] Setting key f78853958a203d19aca2ed2a1c266c17
I20260806 18:44:41.287674 3571 ts_manager.cc:194] Registered new tserver with Master: 57de79e6b48b4a8bbb74d94e70c87552 (127.1.245.65:37827)
I20260806 18:44:41.289033 3571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:50253
I20260806 18:44:41.294838 3884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.287732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33021 (local address 127.1.245.65:48335)
0806 18:44:41.288018 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.288030 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.288111 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.288302 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.288304 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.288497 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:41.288595 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.288600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.289508 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.289511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.290171 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.290180 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.290666 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.290670 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.290760 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.291236 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:41.291247 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:41.291711 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:41.293409 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.293412 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.293413 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.293598 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.293878 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.293880 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.293881 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.293956 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.294403 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:41.294406 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:41.294486 (+ 80us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.294571 (+ 85us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.294677 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":119,"threads_started":1}
I20260806 18:44:41.294930 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.287531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:48335 (local address 127.1.245.126:33021)
0806 18:44:41.287633 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.287637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.288101 (+ 464us) server_negotiation.cc:407] Connection header received
0806 18:44:41.288171 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.288174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.288215 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.288291 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:41.288692 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.289391 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.290281 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.290564 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.290656 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.291851 (+ 1195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:41.291854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:41.291856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:41.291877 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:41.293302 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.293687 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.293691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.293692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.293779 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.294043 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.294046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.294048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.294266 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:41.294400 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.294646 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.294765 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:44:41.318135 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33021
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:41.431296 3889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:41.431578 3889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:41.431622 3889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:41.435635 3889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:41.435698 3889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:41.435738 3889 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:44:41.435771 3889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:44:41.441063 3889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:33021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:41.442140 3889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:41.443181 3889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:41.452700 3894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:41.452670 3896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:41.452662 3900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:41.452725 3889 server_base.cc:1034] running on GCE node
I20260806 18:44:41.453277 3889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:41.453950 3889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:41.455120 3889 hybrid_clock.cc:648] HybridClock initialized: now 1786041881455099 us; error 33 us; skew 500 ppm
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:41.458036 3889 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:44:41.458989 3889 webserver.cc:460] Webserver started at http://127.1.245.66:33595/ using document root <none> and password file <none>
I20260806 18:44:41.459527 3889 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:41.459594 3889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:41.459792 3889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:41.461918 3889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "1b4fe142050c4f22968eaecf120b553a"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "957b3ec1c09be5ffafa3027f497ca28b"
server_key_iv: "f74ae84c670b7126f27586ce4ba8a2e2"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.462600 3889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "1b4fe142050c4f22968eaecf120b553a"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "957b3ec1c09be5ffafa3027f497ca28b"
server_key_iv: "f74ae84c670b7126f27586ce4ba8a2e2"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.469218 3889 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.000s
I20260806 18:44:41.471707 3910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:41.472886 3889 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:44:41.473048 3889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "1b4fe142050c4f22968eaecf120b553a"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "957b3ec1c09be5ffafa3027f497ca28b"
server_key_iv: "f74ae84c670b7126f27586ce4ba8a2e2"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.473155 3889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:41.486238 3889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:41.489424 3889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:41.489589 3889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:41.491165 3889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:41.491231 3889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:41.491277 3889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:41.491305 3889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:41.501947 3889 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:32777
I20260806 18:44:41.502054 4070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:32777 every 8 connection(s)
I20260806 18:44:41.503017 3889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:44:41.504707 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.492069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:35059 (local address 127.1.245.126:33021)
0806 18:44:41.492189 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.492193 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.493282 (+ 1089us) server_negotiation.cc:407] Connection header received
0806 18:44:41.494344 (+ 1062us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.494349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.494416 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.494477 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:41.495918 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.496526 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.497641 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.497892 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.498008 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.500619 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:41.500624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:41.500627 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:41.500656 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:41.502430 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.503045 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.503049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.503051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.503155 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.503507 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.503511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.503513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.503764 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:41.503886 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.504454 (+ 568us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.504579 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:44:41.505458 3920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.492604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33021 (local address 127.1.245.66:35059)
0806 18:44:41.493112 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.493155 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.494058 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.494846 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.494853 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.495181 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:41.495762 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.495779 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.496652 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.496655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.497519 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.497529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.498065 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.498069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.498229 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.498892 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:41.498911 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:41.500442 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:41.502587 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.502592 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.502605 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.502934 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.503245 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.503249 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.503251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.503393 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.503898 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:41.503912 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:41.504158 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.504329 (+ 171us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.504597 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":126,"threads_started":1}
I20260806 18:44:41.506788 4071 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33021
I20260806 18:44:41.506947 4071 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:41.507472 4071 heartbeater.cc:511] Master 127.1.245.126:33021 requested a full tablet report, sending...
I20260806 18:44:41.508632 3572 ts_manager.cc:194] Registered new tserver with Master: 1b4fe142050c4f22968eaecf120b553a (127.1.245.66:32777)
I20260806 18:44:41.508760 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 3889
I20260806 18:44:41.508860 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:44:41.509145 2005 external_mini_cluster.cc:1428] Setting key bf5114ebeab1cfd585892855635688a1
I20260806 18:44:41.509414 3572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:35059
I20260806 18:44:41.516693 4079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.508899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33021 (local address 127.1.245.66:48537)
0806 18:44:41.509222 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.509234 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.509330 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.509564 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.509567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.509922 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:41.510047 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.510054 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.511065 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.511068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.511848 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.511861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.512386 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.512389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.512468 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.512976 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:41.512997 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:41.513359 (+ 362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:41.514949 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.514952 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.514954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.515149 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.515498 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.515502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.515504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.515619 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.516234 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:41.516238 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:41.516339 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.516434 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.516549 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":89,"threads_started":1}
I20260806 18:44:41.516840 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.508973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:48537 (local address 127.1.245.126:33021)
0806 18:44:41.509089 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.509093 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.509305 (+ 212us) server_negotiation.cc:407] Connection header received
0806 18:44:41.509407 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.509410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.509454 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.509525 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:41.510169 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.510911 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.511991 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.512259 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.512367 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.513502 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:41.513506 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:41.513509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:41.513536 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:41.514836 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.515290 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.515294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.515296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.515406 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.515743 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.515747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.515749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.516084 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:41.516185 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.516554 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.516684 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:44:41.537541 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33021
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:41.650501 4083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:41.650756 4083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:41.650816 4083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:41.654816 4083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:41.654881 4083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:41.654933 4083 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:44:41.654990 4083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:44:41.660378 4083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:33021
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:41.661510 4083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:41.662627 4083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:41.672482 4093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:41.672596 4083 server_base.cc:1034] running on GCE node
W20260806 18:44:41.672537 4089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:41.672525 4095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:41.673209 4083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:41.673936 4083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:41.675120 4083 hybrid_clock.cc:648] HybridClock initialized: now 1786041881675079 us; error 54 us; skew 500 ppm
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:41.677855 4083 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:44:41.678869 4083 webserver.cc:460] Webserver started at http://127.1.245.67:46233/ using document root <none> and password file <none>
I20260806 18:44:41.679415 4083 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:41.679473 4083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:41.679634 4083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:41.681643 4083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "6e70b8a5ce8940b5999023fc35691f4d"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "53b28ff27265424affb1f5f022f4b835"
server_key_iv: "421ab06a29c1cd88007d3711d501ce0a"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.682173 4083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "6e70b8a5ce8940b5999023fc35691f4d"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "53b28ff27265424affb1f5f022f4b835"
server_key_iv: "421ab06a29c1cd88007d3711d501ce0a"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.686424 4083 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:41.689049 4106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:41.690389 4083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:44:41.690503 4083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "6e70b8a5ce8940b5999023fc35691f4d"
format_stamp: "Formatted at 2026-08-06 18:44:41 on dist-test-slave-fgb3"
server_key: "53b28ff27265424affb1f5f022f4b835"
server_key_iv: "421ab06a29c1cd88007d3711d501ce0a"
server_key_version: "encryptionkey@0"
I20260806 18:44:41.690613 4083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:41.703110 4083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:41.705940 4083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:41.706118 4083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:41.707737 4083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:41.707788 4083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:41.707834 4083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:41.707863 4083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:41.718030 4083 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:42211
I20260806 18:44:41.718053 4264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:42211 every 8 connection(s)
I20260806 18:44:41.719209 4083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:44:41.721834 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.708631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:57649 (local address 127.1.245.126:33021)
0806 18:44:41.708773 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.708777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.709818 (+ 1041us) server_negotiation.cc:407] Connection header received
0806 18:44:41.710911 (+ 1093us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.710915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.710977 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.711055 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:41.712656 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.713502 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.714988 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.715222 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.715325 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.718098 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:41.718102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:41.718106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:41.718139 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:41.719761 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.720351 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.720355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.720356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.720438 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.720772 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.720776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.720778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.720972 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:41.721060 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.721541 (+ 481us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.721668 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:44:41.722718 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.709149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33021 (local address 127.1.245.67:57649)
0806 18:44:41.709648 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.709698 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.710683 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.711285 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.711294 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.711755 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:41.712455 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.712478 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.713630 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.713634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.714860 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.714871 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.715381 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.715385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.715563 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.716131 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:41.716151 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:41.717858 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:41.719903 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.719909 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.719925 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.720208 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.720537 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.720540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.720543 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.720676 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.721072 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:41.721079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:41.721309 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.721474 (+ 165us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.721789 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":125,"threads_started":1}
I20260806 18:44:41.723968 4265 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33021
I20260806 18:44:41.724130 4265 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:41.724242 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 4083
I20260806 18:44:41.724383 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:44:41.724728 2005 external_mini_cluster.cc:1428] Setting key 7998a5d8584f6860d59bdfda08de921f
I20260806 18:44:41.724725 4265 heartbeater.cc:511] Master 127.1.245.126:33021 requested a full tablet report, sending...
I20260806 18:44:41.725798 3572 ts_manager.cc:194] Registered new tserver with Master: 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211)
I20260806 18:44:41.726550 3572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:57649
I20260806 18:44:41.727125 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:44:41.733848 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.726477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:43071 (local address 127.1.245.126:33021)
0806 18:44:41.726585 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.726589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.727029 (+ 440us) server_negotiation.cc:407] Connection header received
0806 18:44:41.727123 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.727126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.727167 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.727231 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:41.727706 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.728206 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.729045 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.729374 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.729473 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.730645 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:41.730648 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:41.730651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:41.730677 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:41.732136 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.732594 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.732598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.732600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.732691 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.732997 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.733000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.733002 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.733178 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:41.733264 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.733627 (+ 363us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.733731 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:44:41.733865 4276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.726435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33021 (local address 127.1.245.67:43071)
0806 18:44:41.726946 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.726958 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.727047 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.727256 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.727259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.727451 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:41.727602 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.727608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.728334 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.728337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.728937 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.728946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.729482 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.729485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.729562 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.730021 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:41.730031 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:41.730530 (+ 499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:41.732255 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.732257 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.732259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.732506 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.732771 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:41.732773 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:41.732774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:41.732901 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:41.733272 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:41.733276 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:41.733410 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.733521 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.733661 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":18,"thread_start_us":96,"threads_started":1}
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:44:41 dist-test-slave-fgb3 krb5kdc[3509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:41.754742 3642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.746225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58950 (local address 127.1.245.126:33021)
0806 18:44:41.746400 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.746404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.746417 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:44:41.746460 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.746462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.746505 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.746583 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:41.747463 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.747995 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.748856 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.749074 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.749164 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.751497 (+ 2333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:41.751500 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:41.751502 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:41.751532 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:41.752966 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.753413 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.753417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.753418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.753502 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:41.753758 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:41.753761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:41.753762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:41.754053 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:41.754180 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.754460 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.754582 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:44:41.757943 3572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:44:41.761106 3572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:41.774987 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.770018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:42211 (local address 127.0.0.1:35188)
0806 18:44:41.770426 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.770438 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.770562 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.771345 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.771351 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.771373 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:41.771505 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.771511 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.772520 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.772523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.773662 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.773673 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.774449 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.774453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.774560 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.774598 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.774716 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.774839 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260806 18:44:41.775002 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.770218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35188 (local address 127.1.245.67:42211)
0806 18:44:41.770864 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.770876 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.770898 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:44:41.770976 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.770982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.771190 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.771344 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:41.771633 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.772413 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.773785 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.774340 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.774447 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.774627 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.774745 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.774839 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":157,"threads_started":1}
I20260806 18:44:41.775895 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.769598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:32777 (local address 127.0.0.1:53378)
0806 18:44:41.769979 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.770032 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.770841 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.771510 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.771513 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.771530 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:41.771643 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.771650 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.773067 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.773071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.774390 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.774401 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.775493 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.775496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.775602 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.775620 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.775681 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.775736 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":55,"thread_start_us":96,"threads_started":1}
I20260806 18:44:41.775974 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.769700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.1.245.66:32777)
0806 18:44:41.770724 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.770737 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.770762 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:44:41.771005 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.771011 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.771216 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.771350 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:41.771747 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.772940 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.774530 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.775330 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.775434 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.775594 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.775736 (+ 142us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.775829 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":914,"thread_start_us":106,"threads_started":1}
I20260806 18:44:41.775895 4298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.770346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:37827 (local address 127.0.0.1:34490)
0806 18:44:41.771063 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:41.771075 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:41.771147 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:41.771577 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:41.771580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:41.771599 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:41.771681 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.771687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.773067 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.773071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.774391 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:41.774401 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.775457 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.775461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:41.775550 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.775601 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 18:44:41.775663 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:44:41.775736 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":655,"mutex_wait_us":676,"thread_start_us":105,"threads_started":1}
I20260806 18:44:41.777585 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:41.770596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34490 (local address 127.1.245.65:37827)
0806 18:44:41.771169 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:41.771183 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:41.771204 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:44:41.771273 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:41.771278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:41.771449 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:41.771589 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:41.771781 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.772932 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.774530 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:41.775325 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:41.777077 (+ 1752us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:41.777232 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:41.777343 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 18:44:41.777447 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":83,"threads_started":1}
I20260806 18:44:41.778410 4180 tablet_service.cc:1467] Processing CreateTablet for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b (DEFAULT_TABLE table=test-table [id=6aa00bb235d346a3950f831c25ce6ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:41.778810 3978 tablet_service.cc:1467] Processing CreateTablet for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b (DEFAULT_TABLE table=test-table [id=6aa00bb235d346a3950f831c25ce6ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:41.779634 4180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:41.779732 3978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:41.780181 3778 tablet_service.cc:1467] Processing CreateTablet for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b (DEFAULT_TABLE table=test-table [id=6aa00bb235d346a3950f831c25ce6ebb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:41.781133 3778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:41.787446 4299 tablet_bootstrap.cc:492] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Bootstrap starting.
I20260806 18:44:41.788616 4302 tablet_bootstrap.cc:492] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Bootstrap starting.
I20260806 18:44:41.790064 4299 tablet_bootstrap.cc:654] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:41.790338 4302 tablet_bootstrap.cc:654] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:41.790634 4300 tablet_bootstrap.cc:492] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Bootstrap starting.
I20260806 18:44:41.790951 4299 log.cc:824] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:41.791055 4302 log.cc:824] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:41.792918 4300 tablet_bootstrap.cc:654] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:41.793094 4302 tablet_bootstrap.cc:492] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: No bootstrap required, opened a new log
I20260806 18:44:41.793094 4299 tablet_bootstrap.cc:492] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: No bootstrap required, opened a new log
I20260806 18:44:41.793293 4299 ts_tablet_manager.cc:1397] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260806 18:44:41.793303 4302 ts_tablet_manager.cc:1397] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:44:41.793638 4300 log.cc:824] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:41.795878 4300 tablet_bootstrap.cc:492] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: No bootstrap required, opened a new log
I20260806 18:44:41.796056 4300 ts_tablet_manager.cc:1397] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:44:41.797302 4302 raft_consensus.cc:348] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:41.797293 4299 raft_consensus.cc:348] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:41.797564 4302 raft_consensus.cc:374] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:41.797564 4299 raft_consensus.cc:374] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:41.797621 4302 raft_consensus.cc:729] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e70b8a5ce8940b5999023fc35691f4d, State: Initialized, Role: FOLLOWER
I20260806 18:44:41.797621 4299 raft_consensus.cc:729] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4fe142050c4f22968eaecf120b553a, State: Initialized, Role: FOLLOWER
I20260806 18:44:41.798097 4302 consensus_queue.cc:260] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:41.798362 4299 consensus_queue.cc:260] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:41.798899 4265 heartbeater.cc:503] Master 127.1.245.126:33021 was elected leader, sending a full tablet report...
I20260806 18:44:41.799175 4302 ts_tablet_manager.cc:1428] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:44:41.799175 4299 ts_tablet_manager.cc:1428] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260806 18:44:41.799175 4300 raft_consensus.cc:348] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:41.799427 4300 raft_consensus.cc:374] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:41.799474 4300 raft_consensus.cc:729] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57de79e6b48b4a8bbb74d94e70c87552, State: Initialized, Role: FOLLOWER
I20260806 18:44:41.799484 4071 heartbeater.cc:503] Master 127.1.245.126:33021 was elected leader, sending a full tablet report...
I20260806 18:44:41.799863 4300 consensus_queue.cc:260] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:41.800467 3875 heartbeater.cc:503] Master 127.1.245.126:33021 was elected leader, sending a full tablet report...
I20260806 18:44:41.800741 4300 ts_tablet_manager.cc:1428] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260806 18:44:41.970373 4266 tablet.cc:2369] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:44:41.999673 4310 raft_consensus.cc:482] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:41.999882 4310 raft_consensus.cc:504] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.001076 4310 leader_election.cc:288] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1b4fe142050c4f22968eaecf120b553a (127.1.245.66:32777), 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211)
W20260806 18:44:42.004292 4074 tablet.cc:2369] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:44:42.006140 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.001616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:56085 (local address 127.1.245.67:42211)
0806 18:44:42.001789 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.001794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.002038 (+ 244us) server_negotiation.cc:407] Connection header received
0806 18:44:42.002168 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.002171 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.002216 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.002280 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:42.002607 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.003404 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.004790 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.005404 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.005482 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.005528 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.005931 (+ 403us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.006012 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 18:44:42.006140 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.001527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:38241 (local address 127.1.245.66:32777)
0806 18:44:42.001760 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.001765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.001894 (+ 129us) server_negotiation.cc:407] Connection header received
0806 18:44:42.002044 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.002047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.002098 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.002167 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:42.002479 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.003044 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.004781 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.005380 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.005477 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.005528 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.005932 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.006012 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260806 18:44:42.006585 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.001401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:32777 (local address 127.1.245.65:38241)
0806 18:44:42.001787 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:42.001802 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:42.001933 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:42.002188 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:42.002190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:42.002207 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:42.002355 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.002363 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.003149 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.003152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.004295 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.004316 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.005622 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.005626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.005785 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.005808 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:44:42.006283 (+ 475us) client_negotiation.cc:240] Negotiation successful
0806 18:44:42.006409 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"mutex_wait_us":28,"thread_start_us":107,"threads_started":1}
I20260806 18:44:42.006585 4312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.001409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:42211 (local address 127.1.245.65:56085)
0806 18:44:42.001947 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:42.001960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:42.002065 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:42.002296 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:42.002300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:42.002357 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:42.002476 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.002483 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.003540 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.003543 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.004643 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.004653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.005622 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.005626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.005773 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.005829 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 18:44:42.006349 (+ 520us) client_negotiation.cc:240] Negotiation successful
0806 18:44:42.006427 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":130,"thread_start_us":110,"threads_started":1}
I20260806 18:44:42.007225 4207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f7c5706baf4f6ea314f74ec8ab4c9b" candidate_uuid: "57de79e6b48b4a8bbb74d94e70c87552" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e70b8a5ce8940b5999023fc35691f4d" is_pre_election: true
I20260806 18:44:42.007507 4005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f7c5706baf4f6ea314f74ec8ab4c9b" candidate_uuid: "57de79e6b48b4a8bbb74d94e70c87552" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4fe142050c4f22968eaecf120b553a" is_pre_election: true
I20260806 18:44:42.007578 4207 raft_consensus.cc:2454] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57de79e6b48b4a8bbb74d94e70c87552 in term 0.
I20260806 18:44:42.007804 4005 raft_consensus.cc:2454] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 57de79e6b48b4a8bbb74d94e70c87552 in term 0.
I20260806 18:44:42.008127 3711 leader_election.cc:302] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57de79e6b48b4a8bbb74d94e70c87552, 6e70b8a5ce8940b5999023fc35691f4d; no voters:
I20260806 18:44:42.008430 4310 raft_consensus.cc:2781] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:42.008534 4310 raft_consensus.cc:482] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:42.008572 4310 raft_consensus.cc:3037] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.009886 4310 raft_consensus.cc:504] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.010262 4310 leader_election.cc:288] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [CANDIDATE]: Term 1 election: Requested vote from peers 1b4fe142050c4f22968eaecf120b553a (127.1.245.66:32777), 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211)
I20260806 18:44:42.010610 4005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f7c5706baf4f6ea314f74ec8ab4c9b" candidate_uuid: "57de79e6b48b4a8bbb74d94e70c87552" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b4fe142050c4f22968eaecf120b553a"
I20260806 18:44:42.010627 4207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73f7c5706baf4f6ea314f74ec8ab4c9b" candidate_uuid: "57de79e6b48b4a8bbb74d94e70c87552" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e70b8a5ce8940b5999023fc35691f4d"
I20260806 18:44:42.010746 4005 raft_consensus.cc:3037] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.010746 4207 raft_consensus.cc:3037] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.012077 4005 raft_consensus.cc:2454] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57de79e6b48b4a8bbb74d94e70c87552 in term 1.
I20260806 18:44:42.012413 3709 leader_election.cc:302] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4fe142050c4f22968eaecf120b553a, 57de79e6b48b4a8bbb74d94e70c87552; no voters:
I20260806 18:44:42.012632 4310 raft_consensus.cc:2781] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:42.012650 4207 raft_consensus.cc:2454] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 57de79e6b48b4a8bbb74d94e70c87552 in term 1.
I20260806 18:44:42.012879 4310 raft_consensus.cc:686] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 LEADER]: Becoming Leader. State: Replica: 57de79e6b48b4a8bbb74d94e70c87552, State: Running, Role: LEADER
I20260806 18:44:42.013244 4310 consensus_queue.cc:237] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.016669 3572 catalog_manager.cc:5495] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 reported cstate change: term changed from 0 to 1, leader changed from <none> to 57de79e6b48b4a8bbb74d94e70c87552 (127.1.245.65). New cstate: current_term: 1 leader_uuid: "57de79e6b48b4a8bbb74d94e70c87552" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } health_report { overall_health: UNKNOWN } } }
W20260806 18:44:42.033813 3876 tablet.cc:2369] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:44:42.077466 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.073841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34502 (local address 127.1.245.65:37827)
0806 18:44:42.074001 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.074004 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.074020 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:44:42.074177 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.074180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.074236 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.074347 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:42.074624 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.075122 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.075949 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.076158 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.076228 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.076545 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:42.077023 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:42.077108 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.077237 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.077291 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:44:42.083786 4207 raft_consensus.cc:1261] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Refusing update from remote peer 57de79e6b48b4a8bbb74d94e70c87552: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:42.083786 4005 raft_consensus.cc:1261] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Refusing update from remote peer 57de79e6b48b4a8bbb74d94e70c87552: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:42.084515 4313 consensus_queue.cc:1035] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:42.084713 4310 consensus_queue.cc:1035] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:42.092290 4316 mvcc.cc:205] Tried to move back new op lower bound from 7315627549007712256 to 7315627548728938496. Current Snapshot: MvccSnapshot[applied={T|T < 7315627549007712256}]
I20260806 18:44:42.110364 3572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:42.110550 3572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:42.113402 3572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab: Sending DeleteTablet for 3 replicas of tablet 73f7c5706baf4f6ea314f74ec8ab4c9b
I20260806 18:44:42.114283 3978 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:42 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:53378
I20260806 18:44:42.114292 4180 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:42 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35188
I20260806 18:44:42.115178 4334 tablet_replica.cc:330] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: stopping tablet replica
I20260806 18:44:42.115271 3778 tablet_service.cc:1514] Processing DeleteTablet for tablet 73f7c5706baf4f6ea314f74ec8ab4c9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:42 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:34490
I20260806 18:44:42.115461 4334 raft_consensus.cc:2229] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:42.115725 4334 raft_consensus.cc:2258] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:42.116027 4333 tablet_replica.cc:330] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: stopping tablet replica
I20260806 18:44:42.116014 3572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:44:42.116245 4333 raft_consensus.cc:2229] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 18:44:42.116451 3572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:42.116490 4333 raft_consensus.cc:2258] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:42.117873 4334 ts_tablet_manager.cc:1905] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:42.117872 4333 ts_tablet_manager.cc:1905] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:42.120613 4333 ts_tablet_manager.cc:1918] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:44:42.120748 4333 log.cc:1196] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/73f7c5706baf4f6ea314f74ec8ab4c9b
I20260806 18:44:42.120795 4334 ts_tablet_manager.cc:1918] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:44:42.120886 4334 log.cc:1196] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/73f7c5706baf4f6ea314f74ec8ab4c9b
I20260806 18:44:42.121145 4333 ts_tablet_manager.cc:1939] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 1b4fe142050c4f22968eaecf120b553a: Deleting consensus metadata
I20260806 18:44:42.121147 4334 ts_tablet_manager.cc:1939] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 6e70b8a5ce8940b5999023fc35691f4d: Deleting consensus metadata
I20260806 18:44:42.122092 4336 tablet_replica.cc:330] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: stopping tablet replica
I20260806 18:44:42.122439 3553 catalog_manager.cc:4841] TS 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211): tablet 73f7c5706baf4f6ea314f74ec8ab4c9b (table test-table [id=6aa00bb235d346a3950f831c25ce6ebb]) successfully deleted
I20260806 18:44:42.122691 3978 tablet_service.cc:1467] Processing CreateTablet for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 (DEFAULT_TABLE table=test-table [id=687dc48fbf7b4b369a403a7f7f4c517e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:42.122846 4336 raft_consensus.cc:2229] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:44:42.122963 3978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:42.122915 3778 tablet_service.cc:1467] Processing CreateTablet for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 (DEFAULT_TABLE table=test-table [id=687dc48fbf7b4b369a403a7f7f4c517e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:42.123317 3778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:42.123536 4336 raft_consensus.cc:2258] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:42.125085 4299 tablet_bootstrap.cc:492] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: Bootstrap starting.
I20260806 18:44:42.125276 4336 ts_tablet_manager.cc:1905] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:42.125311 3552 catalog_manager.cc:4841] TS 1b4fe142050c4f22968eaecf120b553a (127.1.245.66:32777): tablet 73f7c5706baf4f6ea314f74ec8ab4c9b (table test-table [id=6aa00bb235d346a3950f831c25ce6ebb]) successfully deleted
I20260806 18:44:42.126015 4299 tablet_bootstrap.cc:654] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:42.126116 4300 tablet_bootstrap.cc:492] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: Bootstrap starting.
I20260806 18:44:42.126629 4180 tablet_service.cc:1467] Processing CreateTablet for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 (DEFAULT_TABLE table=test-table [id=687dc48fbf7b4b369a403a7f7f4c517e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:42.126891 4180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:42.127087 4300 tablet_bootstrap.cc:654] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:42.127735 4336 ts_tablet_manager.cc:1918] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:44:42.127871 4336 log.cc:1196] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/73f7c5706baf4f6ea314f74ec8ab4c9b
I20260806 18:44:42.128196 4336 ts_tablet_manager.cc:1939] T 73f7c5706baf4f6ea314f74ec8ab4c9b P 57de79e6b48b4a8bbb74d94e70c87552: Deleting consensus metadata
I20260806 18:44:42.128298 4299 tablet_bootstrap.cc:492] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: No bootstrap required, opened a new log
I20260806 18:44:42.128388 4299 ts_tablet_manager.cc:1397] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 18:44:42.128834 4299 raft_consensus.cc:348] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.128950 4299 raft_consensus.cc:374] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:42.128986 4299 raft_consensus.cc:729] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4fe142050c4f22968eaecf120b553a, State: Initialized, Role: FOLLOWER
I20260806 18:44:42.129132 4302 tablet_bootstrap.cc:492] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: Bootstrap starting.
I20260806 18:44:42.129539 3552 catalog_manager.cc:4841] TS 57de79e6b48b4a8bbb74d94e70c87552 (127.1.245.65:37827): tablet 73f7c5706baf4f6ea314f74ec8ab4c9b (table test-table [id=6aa00bb235d346a3950f831c25ce6ebb]) successfully deleted
I20260806 18:44:42.130038 4302 tablet_bootstrap.cc:654] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:42.130247 4299 consensus_queue.cc:260] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.130892 4300 tablet_bootstrap.cc:492] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: No bootstrap required, opened a new log
I20260806 18:44:42.130972 4300 ts_tablet_manager.cc:1397] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 18:44:42.131409 4300 raft_consensus.cc:348] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.131518 4300 raft_consensus.cc:374] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:42.131546 4300 raft_consensus.cc:729] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57de79e6b48b4a8bbb74d94e70c87552, State: Initialized, Role: FOLLOWER
I20260806 18:44:42.131656 4300 consensus_queue.cc:260] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.131697 4302 tablet_bootstrap.cc:492] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: No bootstrap required, opened a new log
I20260806 18:44:42.131780 4302 ts_tablet_manager.cc:1397] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 18:44:42.131918 4300 ts_tablet_manager.cc:1428] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:42.132251 4299 ts_tablet_manager.cc:1428] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260806 18:44:42.132200 4302 raft_consensus.cc:348] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.132304 4302 raft_consensus.cc:374] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:42.132337 4302 raft_consensus.cc:729] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e70b8a5ce8940b5999023fc35691f4d, State: Initialized, Role: FOLLOWER
I20260806 18:44:42.132468 4302 consensus_queue.cc:260] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.132700 4302 ts_tablet_manager.cc:1428] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:44:42.329063 4309 raft_consensus.cc:482] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:42.329221 4309 raft_consensus.cc:504] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.330459 4309 leader_election.cc:288] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57de79e6b48b4a8bbb74d94e70c87552 (127.1.245.65:37827), 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211)
I20260806 18:44:42.334408 4341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.330646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:37827 (local address 127.1.245.66:52315)
0806 18:44:42.330962 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:42.330975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:42.331096 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:42.331342 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:42.331345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:42.331361 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:42.331485 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.331492 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.332433 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.332436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.333272 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.333282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.333958 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.333962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.334064 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.334087 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:44:42.334152 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:44:42.334224 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":83,"threads_started":1}
I20260806 18:44:42.334396 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.330740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:52315 (local address 127.1.245.65:37827)
0806 18:44:42.330882 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.330887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.331059 (+ 172us) server_negotiation.cc:407] Connection header received
0806 18:44:42.331203 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.331206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.331262 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.331325 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:42.331611 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.332335 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.333403 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.333828 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.333942 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.334004 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.334179 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.334268 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:44:42.334610 4296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.331506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:49629 (local address 127.1.245.67:42211)
0806 18:44:42.331656 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.331661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.331673 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:44:42.331719 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.331722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.331766 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.331844 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:42.332055 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.332666 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.333647 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.334046 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.334149 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.334200 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.334408 (+ 208us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.334479 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:44:42.334944 4342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.330808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:42211 (local address 127.1.245.66:49629)
0806 18:44:42.331128 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:42.331142 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:42.331239 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:42.331856 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:42.331858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:42.331875 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:42.331965 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.331969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.332771 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.332774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.333528 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.333536 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.334149 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.334153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.334278 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.334296 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:44:42.334389 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:44:42.334829 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":19,"thread_start_us":136,"threads_started":1}
I20260806 18:44:42.334914 3802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "581bf8e35c9c4f8aaaa7cdc4036e6565" candidate_uuid: "1b4fe142050c4f22968eaecf120b553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57de79e6b48b4a8bbb74d94e70c87552" is_pre_election: true
I20260806 18:44:42.335155 3802 raft_consensus.cc:2454] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4fe142050c4f22968eaecf120b553a in term 0.
I20260806 18:44:42.335369 4207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "581bf8e35c9c4f8aaaa7cdc4036e6565" candidate_uuid: "1b4fe142050c4f22968eaecf120b553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e70b8a5ce8940b5999023fc35691f4d" is_pre_election: true
I20260806 18:44:42.335515 4207 raft_consensus.cc:2454] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4fe142050c4f22968eaecf120b553a in term 0.
I20260806 18:44:42.335601 3913 leader_election.cc:302] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4fe142050c4f22968eaecf120b553a, 57de79e6b48b4a8bbb74d94e70c87552; no voters:
I20260806 18:44:42.335897 4309 raft_consensus.cc:2781] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:42.335963 4309 raft_consensus.cc:482] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:42.336000 4309 raft_consensus.cc:3037] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.336982 4309 raft_consensus.cc:504] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.337340 4309 leader_election.cc:288] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [CANDIDATE]: Term 1 election: Requested vote from peers 57de79e6b48b4a8bbb74d94e70c87552 (127.1.245.65:37827), 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211)
I20260806 18:44:42.337695 3802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "581bf8e35c9c4f8aaaa7cdc4036e6565" candidate_uuid: "1b4fe142050c4f22968eaecf120b553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57de79e6b48b4a8bbb74d94e70c87552"
I20260806 18:44:42.337762 4207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "581bf8e35c9c4f8aaaa7cdc4036e6565" candidate_uuid: "1b4fe142050c4f22968eaecf120b553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e70b8a5ce8940b5999023fc35691f4d"
I20260806 18:44:42.337854 3802 raft_consensus.cc:3037] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.337910 4207 raft_consensus.cc:3037] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.339116 4207 raft_consensus.cc:2454] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4fe142050c4f22968eaecf120b553a in term 1.
I20260806 18:44:42.339116 3802 raft_consensus.cc:2454] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4fe142050c4f22968eaecf120b553a in term 1.
I20260806 18:44:42.339447 3914 leader_election.cc:302] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4fe142050c4f22968eaecf120b553a, 6e70b8a5ce8940b5999023fc35691f4d; no voters:
I20260806 18:44:42.339617 4309 raft_consensus.cc:2781] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:42.339902 4309 raft_consensus.cc:686] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 1 LEADER]: Becoming Leader. State: Replica: 1b4fe142050c4f22968eaecf120b553a, State: Running, Role: LEADER
I20260806 18:44:42.340169 4309 consensus_queue.cc:237] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.342559 3572 catalog_manager.cc:5495] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b4fe142050c4f22968eaecf120b553a (127.1.245.66). New cstate: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } health_report { overall_health: UNKNOWN } } }
Aug 06 18:44:42 dist-test-slave-fgb3 krb5kdc[3509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041880, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:42.396651 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.388268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33021 (local address 127.0.0.1:58966)
0806 18:44:42.388595 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:42.388611 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:42.388737 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:42.389014 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:42.389017 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:42.389495 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:42.389606 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.389614 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.390377 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.390380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.390933 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.390942 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.391339 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.391342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.391462 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.391985 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:42.391998 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:42.393387 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:42.395113 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:42.395122 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:42.395128 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:42.395398 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:42.395656 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:42.395658 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:42.395660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:42.395733 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:42.396103 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:42.396109 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:42.396223 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 18:44:42.396330 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:44:42.396458 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":31,"thread_start_us":108,"threads_started":1}
I20260806 18:44:42.396653 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.388320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.1.245.126:33021)
0806 18:44:42.388599 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.388608 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.388685 (+ 77us) server_negotiation.cc:407] Connection header received
0806 18:44:42.388821 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.388824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.388887 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.388961 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:42.389725 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.390258 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.391037 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.391246 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.391330 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.393509 (+ 2179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:42.393514 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:42.393516 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:42.393541 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:42.395011 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:42.395482 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:42.395485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:42.395486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:42.395578 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:42.395802 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:42.395804 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:42.395805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:42.395987 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:42.396060 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.396372 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.396516 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":228,"thread_start_us":137,"threads_started":1}
I20260806 18:44:42.400792 3572 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:58966:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:44:42.401592 3572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:42.406518 4180 tablet_service.cc:1467] Processing CreateTablet for tablet f3fe5fd92bae4af9b80de03bcb9ed9cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ebcbb68c204641ea8900ca8a1b72444b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:44:42.406508 3978 tablet_service.cc:1467] Processing CreateTablet for tablet f3fe5fd92bae4af9b80de03bcb9ed9cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ebcbb68c204641ea8900ca8a1b72444b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:44:42.406493 3778 tablet_service.cc:1467] Processing CreateTablet for tablet f3fe5fd92bae4af9b80de03bcb9ed9cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ebcbb68c204641ea8900ca8a1b72444b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:44:42.406976 3778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3fe5fd92bae4af9b80de03bcb9ed9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:42.406977 4180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3fe5fd92bae4af9b80de03bcb9ed9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:42.406977 3978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f3fe5fd92bae4af9b80de03bcb9ed9cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:42.409636 4300 tablet_bootstrap.cc:492] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: Bootstrap starting.
I20260806 18:44:42.409795 4299 tablet_bootstrap.cc:492] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: Bootstrap starting.
I20260806 18:44:42.409910 4302 tablet_bootstrap.cc:492] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: Bootstrap starting.
I20260806 18:44:42.410561 4300 tablet_bootstrap.cc:654] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:42.410670 4299 tablet_bootstrap.cc:654] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:42.410874 4302 tablet_bootstrap.cc:654] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:42.411695 4300 tablet_bootstrap.cc:492] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: No bootstrap required, opened a new log
I20260806 18:44:42.411801 4300 ts_tablet_manager.cc:1397] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:42.411968 4302 tablet_bootstrap.cc:492] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: No bootstrap required, opened a new log
I20260806 18:44:42.411993 4299 tablet_bootstrap.cc:492] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: No bootstrap required, opened a new log
I20260806 18:44:42.412045 4302 ts_tablet_manager.cc:1397] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260806 18:44:42.412063 4299 ts_tablet_manager.cc:1397] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 18:44:42.412228 4300 raft_consensus.cc:348] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.412343 4300 raft_consensus.cc:374] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:42.412384 4300 raft_consensus.cc:729] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57de79e6b48b4a8bbb74d94e70c87552, State: Initialized, Role: FOLLOWER
I20260806 18:44:42.412425 4302 raft_consensus.cc:348] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.412477 4299 raft_consensus.cc:348] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.412539 4302 raft_consensus.cc:374] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:42.412511 4300 consensus_queue.cc:260] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.412572 4302 raft_consensus.cc:729] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e70b8a5ce8940b5999023fc35691f4d, State: Initialized, Role: FOLLOWER
I20260806 18:44:42.412575 4299 raft_consensus.cc:374] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:42.412607 4299 raft_consensus.cc:729] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b4fe142050c4f22968eaecf120b553a, State: Initialized, Role: FOLLOWER
I20260806 18:44:42.412686 4302 consensus_queue.cc:260] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.412724 4299 consensus_queue.cc:260] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.412801 4300 ts_tablet_manager.cc:1428] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:42.412940 4302 ts_tablet_manager.cc:1428] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:42.412978 4299 ts_tablet_manager.cc:1428] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:44:42.413067 4310 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.413200 4310 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:42.413153 4309 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.413249 4309 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:42.413187 4308 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.413316 4308 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:42.474105 4309 raft_consensus.cc:482] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:42.474293 4309 raft_consensus.cc:504] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.474751 4309 leader_election.cc:288] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57de79e6b48b4a8bbb74d94e70c87552 (127.1.245.65:37827), 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211)
I20260806 18:44:42.475222 3802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3fe5fd92bae4af9b80de03bcb9ed9cb" candidate_uuid: "1b4fe142050c4f22968eaecf120b553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57de79e6b48b4a8bbb74d94e70c87552" is_pre_election: true
I20260806 18:44:42.475231 4207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3fe5fd92bae4af9b80de03bcb9ed9cb" candidate_uuid: "1b4fe142050c4f22968eaecf120b553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e70b8a5ce8940b5999023fc35691f4d" is_pre_election: true
I20260806 18:44:42.475396 4207 raft_consensus.cc:2454] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4fe142050c4f22968eaecf120b553a in term 0.
I20260806 18:44:42.475405 3802 raft_consensus.cc:2454] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b4fe142050c4f22968eaecf120b553a in term 0.
I20260806 18:44:42.475701 3914 leader_election.cc:302] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4fe142050c4f22968eaecf120b553a, 6e70b8a5ce8940b5999023fc35691f4d; no voters:
I20260806 18:44:42.475885 4309 raft_consensus.cc:2781] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:42.475948 4309 raft_consensus.cc:482] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:42.475980 4309 raft_consensus.cc:3037] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.477012 4309 raft_consensus.cc:504] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.477332 4309 leader_election.cc:288] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [CANDIDATE]: Term 1 election: Requested vote from peers 57de79e6b48b4a8bbb74d94e70c87552 (127.1.245.65:37827), 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211)
I20260806 18:44:42.477669 4207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3fe5fd92bae4af9b80de03bcb9ed9cb" candidate_uuid: "1b4fe142050c4f22968eaecf120b553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e70b8a5ce8940b5999023fc35691f4d"
I20260806 18:44:42.477784 4207 raft_consensus.cc:3037] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.477715 3802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f3fe5fd92bae4af9b80de03bcb9ed9cb" candidate_uuid: "1b4fe142050c4f22968eaecf120b553a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57de79e6b48b4a8bbb74d94e70c87552"
I20260806 18:44:42.477937 3802 raft_consensus.cc:3037] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:42.478761 4207 raft_consensus.cc:2454] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4fe142050c4f22968eaecf120b553a in term 1.
I20260806 18:44:42.479089 3802 raft_consensus.cc:2454] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b4fe142050c4f22968eaecf120b553a in term 1.
I20260806 18:44:42.479079 3914 leader_election.cc:302] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b4fe142050c4f22968eaecf120b553a, 6e70b8a5ce8940b5999023fc35691f4d; no voters:
I20260806 18:44:42.479285 4309 raft_consensus.cc:2781] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:42.479393 4309 raft_consensus.cc:686] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [term 1 LEADER]: Becoming Leader. State: Replica: 1b4fe142050c4f22968eaecf120b553a, State: Running, Role: LEADER
I20260806 18:44:42.479537 4309 consensus_queue.cc:237] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } }
I20260806 18:44:42.480285 4343 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: TxnStatusTablet state changed. Reason: New leader 1b4fe142050c4f22968eaecf120b553a. Latest consensus state: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.480418 4343 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:44:42.480803 4359 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:44:42.481007 4359 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:44:42.481477 3570 catalog_manager.cc:5495] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b4fe142050c4f22968eaecf120b553a (127.1.245.66). New cstate: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } health_report { overall_health: UNKNOWN } } }
I20260806 18:44:42.496615 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.492287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:32777 (local address 127.0.0.1:53388)
0806 18:44:42.492453 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:42.492469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:42.492566 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:42.492959 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:42.492962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:42.492970 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:44:42.493069 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.493074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.494049 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.494052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.494790 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.494802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.495422 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.495425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.495495 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.495546 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:44:42.496342 (+ 796us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:44:42.496359 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:44:42.496419 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:44:42.496497 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:44:42.496645 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.492417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.1.245.66:32777)
0806 18:44:42.492559 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.492566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.492584 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:44:42.492729 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.492733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.492806 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.492919 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:42.493172 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.493922 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.494919 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.495301 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.495423 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.495680 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:42.496213 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:42.496338 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.496437 (+ 99us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.496498 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:44:42.499920 4207 raft_consensus.cc:1261] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Refusing update from remote peer 1b4fe142050c4f22968eaecf120b553a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:42.500119 3802 raft_consensus.cc:1261] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Refusing update from remote peer 1b4fe142050c4f22968eaecf120b553a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:42.500718 4343 consensus_queue.cc:1035] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:42.500910 4309 consensus_queue.cc:1035] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:42.504074 4310 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: TxnStatusTablet state changed. Reason: New leader 1b4fe142050c4f22968eaecf120b553a. Latest consensus state: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.504200 4310 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:42.506268 4343 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.506393 4343 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:44:42.507555 4308 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: TxnStatusTablet state changed. Reason: New leader 1b4fe142050c4f22968eaecf120b553a. Latest consensus state: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.507684 4308 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:42.508713 4318 mvcc.cc:205] Tried to move back new op lower bound from 7315627550714617856 to 7315627550637584384. Current Snapshot: MvccSnapshot[applied={T|T < 7315627550714617856}]
I20260806 18:44:42.509290 4317 mvcc.cc:205] Tried to move back new op lower bound from 7315627550714617856 to 7315627550637584384. Current Snapshot: MvccSnapshot[applied={T|T < 7315627550714617856 or (T in {7315627550714617856})}]
I20260806 18:44:42.509652 4308 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.509758 4308 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 6e70b8a5ce8940b5999023fc35691f4d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:42.513018 4343 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.513125 4343 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 1b4fe142050c4f22968eaecf120b553a: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:44:42.513962 4310 tablet_replica.cc:439] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b4fe142050c4f22968eaecf120b553a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 } } peers { permanent_uuid: "1b4fe142050c4f22968eaecf120b553a" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 32777 } } peers { permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 } } }
I20260806 18:44:42.514077 4310 tablet_replica.cc:442] T f3fe5fd92bae4af9b80de03bcb9ed9cb P 57de79e6b48b4a8bbb74d94e70c87552: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:42.518190 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.511926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.1.245.66:32777)
0806 18:44:42.512037 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.512042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.512058 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:44:42.512104 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.512107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.512158 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.512240 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:42.514894 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.515656 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.516855 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.517185 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.517523 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.517585 (+ 62us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:42.517658 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:42.517997 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.518027 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.518076 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:44:42.529681 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.525607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:45149 (local address 127.1.245.66:32777)
0806 18:44:42.525711 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.525715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.525969 (+ 254us) server_negotiation.cc:407] Connection header received
0806 18:44:42.526077 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.526080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.526121 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.526181 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:42.527027 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.527507 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.528761 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.528961 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.529039 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.529323 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:42.529383 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:42.529431 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.529526 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.529577 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":33}
I20260806 18:44:42.530025 4372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.525532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:32777 (local address 127.1.245.66:45149)
0806 18:44:42.525888 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:42.525902 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:42.526004 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:42.526205 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:42.526208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:42.526216 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:44:42.526907 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.526915 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.527603 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.527606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.528627 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:42.528638 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.529061 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.529064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:42.529157 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.529208 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:44:42.529452 (+ 244us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:44:42.529458 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:44:42.529521 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:44:42.529624 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":35,"thread_start_us":98,"threads_started":1}
I20260806 18:44:42.537345 4207 raft_consensus.cc:1261] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Refusing update from remote peer 1b4fe142050c4f22968eaecf120b553a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:42.537449 3802 raft_consensus.cc:1261] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Refusing update from remote peer 1b4fe142050c4f22968eaecf120b553a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:42.537811 4309 consensus_queue.cc:1035] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e70b8a5ce8940b5999023fc35691f4d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 42211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:42.537951 4343 consensus_queue.cc:1035] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [LEADER]: Connected to new peer: Peer: permanent_uuid: "57de79e6b48b4a8bbb74d94e70c87552" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:42.575241 3570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:42.575435 3570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:42.577699 3570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a6f54db4b7ec4f249663939007cad5ab: Sending DeleteTablet for 3 replicas of tablet 581bf8e35c9c4f8aaaa7cdc4036e6565
I20260806 18:44:42.578382 3778 tablet_service.cc:1514] Processing DeleteTablet for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:42 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:34490
I20260806 18:44:42.578446 3978 tablet_service.cc:1514] Processing DeleteTablet for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:42 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:53378
I20260806 18:44:42.578570 4336 tablet_replica.cc:330] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: stopping tablet replica
I20260806 18:44:42.578649 4333 tablet_replica.cc:330] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: stopping tablet replica
I20260806 18:44:42.578682 4336 raft_consensus.cc:2229] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:42.578759 4333 raft_consensus.cc:2229] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:44:42.578800 4336 raft_consensus.cc:2258] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:42.579090 4333 raft_consensus.cc:2258] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:42.579908 4333 ts_tablet_manager.cc:1905] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:42.580461 4336 ts_tablet_manager.cc:1905] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:42.582199 4333 ts_tablet_manager.cc:1918] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:42.582270 4333 log.cc:1196] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/581bf8e35c9c4f8aaaa7cdc4036e6565
I20260806 18:44:42.582516 4333 ts_tablet_manager.cc:1939] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 1b4fe142050c4f22968eaecf120b553a: Deleting consensus metadata
I20260806 18:44:42.583189 4336 ts_tablet_manager.cc:1918] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:42.583191 3552 catalog_manager.cc:4841] TS 1b4fe142050c4f22968eaecf120b553a (127.1.245.66:32777): tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 (table test-table [id=687dc48fbf7b4b369a403a7f7f4c517e]) successfully deleted
I20260806 18:44:42.583264 4336 log.cc:1196] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/581bf8e35c9c4f8aaaa7cdc4036e6565
I20260806 18:44:42.583245 4180 tablet_service.cc:1514] Processing DeleteTablet for tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:42 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35188
I20260806 18:44:42.583426 4334 tablet_replica.cc:330] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: stopping tablet replica
I20260806 18:44:42.583493 4336 ts_tablet_manager.cc:1939] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 57de79e6b48b4a8bbb74d94e70c87552: Deleting consensus metadata
I20260806 18:44:42.583542 4334 raft_consensus.cc:2229] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:42.583664 4334 raft_consensus.cc:2258] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:42.584173 3552 catalog_manager.cc:4841] TS 57de79e6b48b4a8bbb74d94e70c87552 (127.1.245.65:37827): tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 (table test-table [id=687dc48fbf7b4b369a403a7f7f4c517e]) successfully deleted
I20260806 18:44:42.584347 4334 ts_tablet_manager.cc:1905] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:42.587738 4334 ts_tablet_manager.cc:1918] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:42.587810 4334 log.cc:1196] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/581bf8e35c9c4f8aaaa7cdc4036e6565
I20260806 18:44:42.587980 4334 ts_tablet_manager.cc:1939] T 581bf8e35c9c4f8aaaa7cdc4036e6565 P 6e70b8a5ce8940b5999023fc35691f4d: Deleting consensus metadata
I20260806 18:44:42.588828 3553 catalog_manager.cc:4841] TS 6e70b8a5ce8940b5999023fc35691f4d (127.1.245.67:42211): tablet 581bf8e35c9c4f8aaaa7cdc4036e6565 (table test-table [id=687dc48fbf7b4b369a403a7f7f4c517e]) successfully deleted
Aug 06 18:44:42 dist-test-slave-fgb3 krb5kdc[3509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041881, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:44:42.597237 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.584437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34516 (local address 127.1.245.65:37827)
0806 18:44:42.586515 (+ 2078us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.586520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.586548 (+ 28us) server_negotiation.cc:407] Connection header received
0806 18:44:42.586588 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.586591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.586637 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.586715 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:42.587751 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.588585 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.589628 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.589983 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.590093 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.593251 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:42.593263 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:42.593277 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:42.593312 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:42.595336 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:42.595824 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:42.595830 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:42.595832 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:42.595932 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:42.596206 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:42.596209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:42.596211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:42.596542 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:42.596759 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.596942 (+ 183us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.597068 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1909}
W20260806 18:44:42.597862 3862 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34516
I20260806 18:44:42.606897 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:42.599757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58968 (local address 127.1.245.126:33021)
0806 18:44:42.599925 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:42.599930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:42.599946 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:44:42.600023 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:42.600026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:42.600072 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:42.600137 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:42.600766 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.601332 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.602099 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:42.602411 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:42.602505 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:42.603867 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:42.603871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:42.603874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:42.603901 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:42.605254 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:42.605658 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:42.605662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:42.605664 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:42.605747 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:42.605991 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:42.605995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:42.605996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:42.606178 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:42.606342 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:42.606576 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:44:42.606729 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260806 18:44:42.607323 3570 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58968
I20260806 18:44:42.608981 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 3676
I20260806 18:44:42.628399 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 3889
I20260806 18:44:42.645367 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 4083
I20260806 18:44:42.654260 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 3528
2026-08-06T18:44:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3068 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:42 dist-test-slave-fgb3 krb5kdc[4386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:42 dist-test-slave-fgb3 krb5kdc[4386](info): set up 2 sockets
Aug 06 18:44:42 dist-test-slave-fgb3 krb5kdc[4386](info): commencing operation
krb5kdc: starting...
W20260806 18:44:43.791170 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:43 dist-test-slave-fgb3 krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:43.922729 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:45825
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:45825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:44.039039 4402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:44.039358 4402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:44.039419 4402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:44.043498 4402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:44.043565 4402 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:44:44.043588 4402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:44.043612 4402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:44.043632 4402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:44.043654 4402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:44.049196 4402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:45825
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:45825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:44.050388 4402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:44.051503 4402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:44.060447 4409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:44.060485 4408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:44.060447 4411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:44.060750 4402 server_base.cc:1034] running on GCE node
I20260806 18:44:44.061457 4402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:44.062744 4402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:44.063947 4402 hybrid_clock.cc:648] HybridClock initialized: now 1786041884063918 us; error 44 us; skew 500 ppm
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:44.067040 4402 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:44.068009 4402 webserver.cc:460] Webserver started at http://127.1.245.126:43193/ using document root <none> and password file <none>
I20260806 18:44:44.068576 4402 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:44.068640 4402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:44.068861 4402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:44.071059 4402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "509abd8a25424c798ca710e94e389b36"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "7ebd18177c1457598ea18e1b7131df37"
server_key_iv: "b2bb9588c263bcc4af14b103cb4c8aa6"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.071630 4402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "509abd8a25424c798ca710e94e389b36"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "7ebd18177c1457598ea18e1b7131df37"
server_key_iv: "b2bb9588c263bcc4af14b103cb4c8aa6"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.075459 4402 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 18:44:44.077948 4423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:44.079316 4402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:44:44.079437 4402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "509abd8a25424c798ca710e94e389b36"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "7ebd18177c1457598ea18e1b7131df37"
server_key_iv: "b2bb9588c263bcc4af14b103cb4c8aa6"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.079555 4402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:44.098070 4402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:44.101182 4402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:44.101379 4402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:44.109494 4402 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:45825
I20260806 18:44:44.109500 4516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:45825 every 8 connection(s)
I20260806 18:44:44.110627 4402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:44.113708 4517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:44.119702 4517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: Bootstrap starting.
I20260806 18:44:44.120060 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 4402
I20260806 18:44:44.120178 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:44.120466 2005 external_mini_cluster.cc:1428] Setting key 5497323d563e7d73a48ba4315b1bf51d
I20260806 18:44:44.122673 4517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:44.123477 4517 log.cc:824] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:44.125855 4517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: No bootstrap required, opened a new log
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:44.130548 4517 raft_consensus.cc:348] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509abd8a25424c798ca710e94e389b36" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45825 } }
I20260806 18:44:44.130892 4517 raft_consensus.cc:374] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:44.130970 4517 raft_consensus.cc:729] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 509abd8a25424c798ca710e94e389b36, State: Initialized, Role: FOLLOWER
I20260806 18:44:44.131515 4517 consensus_queue.cc:260] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509abd8a25424c798ca710e94e389b36" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45825 } }
I20260806 18:44:44.131690 4517 raft_consensus.cc:388] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:44.131763 4517 raft_consensus.cc:482] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:44.131866 4517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:44.132849 4517 raft_consensus.cc:504] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509abd8a25424c798ca710e94e389b36" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45825 } }
I20260806 18:44:44.133210 4517 leader_election.cc:302] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 509abd8a25424c798ca710e94e389b36; no voters:
I20260806 18:44:44.133543 4517 leader_election.cc:288] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:44.133673 4527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:44.133949 4527 raft_consensus.cc:686] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [term 1 LEADER]: Becoming Leader. State: Replica: 509abd8a25424c798ca710e94e389b36, State: Running, Role: LEADER
I20260806 18:44:44.134281 4527 consensus_queue.cc:237] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509abd8a25424c798ca710e94e389b36" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45825 } }
I20260806 18:44:44.135035 4517 sys_catalog.cc:564] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:44.136209 4529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 509abd8a25424c798ca710e94e389b36. Latest consensus state: current_term: 1 leader_uuid: "509abd8a25424c798ca710e94e389b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509abd8a25424c798ca710e94e389b36" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45825 } } }
I20260806 18:44:44.136343 4529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:44.136498 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.121989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50378 (local address 127.1.245.126:45825)
0806 18:44:44.122550 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.122566 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.122606 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:44:44.123370 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.123392 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.123784 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.124182 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:44.125044 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.126107 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.127331 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.127910 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.128049 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.130991 (+ 2942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:44.130999 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:44.131017 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:44.131060 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:44.132964 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.133437 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.133442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.133448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.133601 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.133986 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.133989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.133991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.134574 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:44.134773 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.135090 (+ 317us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.135421 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":158,"threads_started":1}
I20260806 18:44:44.136695 4528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "509abd8a25424c798ca710e94e389b36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "509abd8a25424c798ca710e94e389b36" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45825 } } }
I20260806 18:44:44.136786 4528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:44.137293 4537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:44.141037 4537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:44.146505 4537 catalog_manager.cc:1347] Generated new cluster ID: 03c167d08e86415ea5b0520ded9c9b5e
I20260806 18:44:44.146584 4537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:44.158555 4537 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:44.159946 4537 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:44.165529 4537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: Generated new TSK 0
I20260806 18:44:44.166430 4537 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:44:44.205978 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45825
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:44.322642 4556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:44.322921 4556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:44.322979 4556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:44.327159 4556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:44.327235 4556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:44.327282 4556 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:44:44.327316 4556 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:44:44.327376 4556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:44:44.332850 4556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:44.334196 4556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:44.335575 4556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:44.345944 4565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:44.345948 4564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:44.346027 4556 server_base.cc:1034] running on GCE node
W20260806 18:44:44.345927 4568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:44.346619 4556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:44.347265 4556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:44.348438 4556 hybrid_clock.cc:648] HybridClock initialized: now 1786041884348416 us; error 34 us; skew 500 ppm
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:44.351241 4556 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:44:44.352106 4556 webserver.cc:460] Webserver started at http://127.1.245.65:43223/ using document root <none> and password file <none>
I20260806 18:44:44.352670 4556 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:44.352754 4556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:44.352990 4556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:44.355067 4556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "52be427786d247159029b3cdf492b0c0"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "042f79f90d938e70e38e565dab53770f"
server_key_iv: "875423fd51687a432a1945c396458cef"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.355602 4556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "52be427786d247159029b3cdf492b0c0"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "042f79f90d938e70e38e565dab53770f"
server_key_iv: "875423fd51687a432a1945c396458cef"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.359339 4556 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:44.361969 4578 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:44.363322 4556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:44:44.363448 4556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "52be427786d247159029b3cdf492b0c0"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "042f79f90d938e70e38e565dab53770f"
server_key_iv: "875423fd51687a432a1945c396458cef"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.363562 4556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:44.377249 4556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:44.380699 4556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:44.380899 4556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:44.382668 4556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:44.382731 4556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:44.382783 4556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:44.382803 4556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:44.393889 4556 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:34183
I20260806 18:44:44.393940 4743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:34183 every 8 connection(s)
I20260806 18:44:44.395066 4556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:44:44.397574 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.383478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:57039 (local address 127.1.245.126:45825)
0806 18:44:44.383635 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.383640 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.384699 (+ 1059us) server_negotiation.cc:407] Connection header received
0806 18:44:44.385971 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.385976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.386046 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.386108 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:44.388068 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.388744 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.390553 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.390846 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.390946 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.394018 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:44.394022 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:44.394025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:44.394057 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:44.395658 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.396193 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.396196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.396197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.396281 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.396536 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.396538 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.396539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.396716 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:44.396786 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.397333 (+ 547us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.397449 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260806 18:44:44.398409 4590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.383947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45825 (local address 127.1.245.65:57039)
0806 18:44:44.384464 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.384529 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.385676 (+ 1147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.386623 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.386633 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.387133 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:44.387888 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.387912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.388866 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.388870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.390406 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.390420 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.391027 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.391032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.391249 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.391929 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:44.391954 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:44.393816 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:44.395810 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.395816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.395833 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.396111 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.396347 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.396348 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.396350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.396468 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.396818 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:44.396823 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:44.397058 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.397216 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.397476 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":99,"threads_started":1}
I20260806 18:44:44.399915 4744 heartbeater.cc:347] Connected to a master server at 127.1.245.126:45825
I20260806 18:44:44.400063 4744 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:44.400588 4744 heartbeater.cc:511] Master 127.1.245.126:45825 requested a full tablet report, sending...
I20260806 18:44:44.402262 4450 ts_manager.cc:194] Registered new tserver with Master: 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183)
I20260806 18:44:44.403475 4450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:57039
I20260806 18:44:44.403678 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 4556
I20260806 18:44:44.403793 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:44:44.404157 2005 external_mini_cluster.cc:1428] Setting key 2e0553d327b9a45ac9a47c7781795d25
I20260806 18:44:44.411334 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.402363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:54531 (local address 127.1.245.126:45825)
0806 18:44:44.403792 (+ 1429us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.403797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.403811 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:44:44.403846 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.403849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.403889 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.403972 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:44.404563 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.405303 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.406383 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.406693 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.406823 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.408013 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:44.408016 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:44.408018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:44.408043 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:44.409562 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.410048 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.410051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.410052 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.410135 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.410463 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.410465 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.410466 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.410648 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:44.410756 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.411086 (+ 330us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.411198 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1352}
I20260806 18:44:44.411355 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.402190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45825 (local address 127.1.245.65:54531)
0806 18:44:44.402593 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.402606 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.402689 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.404024 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.404027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.404268 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:44.404422 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.404432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.405458 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.405462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.406225 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.406235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.406833 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.406836 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.406911 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.407402 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:44.407417 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:44.407856 (+ 439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:44.409695 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.409698 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.409700 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.409948 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.410233 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.410237 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.410239 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.410374 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.410757 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:44.410761 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:44.410871 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.410992 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.411143 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:44:44.434602 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45825
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:44.549662 4758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:44.549937 4758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:44.550004 4758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:44.554060 4758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:44.554121 4758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:44.554174 4758 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:44:44.554194 4758 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:44:44.554251 4758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:44:44.559748 4758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:44.560853 4758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:44.561903 4758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:44:44.571969 4758 server_base.cc:1034] running on GCE node
W20260806 18:44:44.571924 4763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:44.571924 4767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:44.571945 4765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:44.572744 4758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:44.573530 4758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:44.574697 4758 hybrid_clock.cc:648] HybridClock initialized: now 1786041884574653 us; error 53 us; skew 500 ppm
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:44.580941 4758 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:44:44.581972 4758 webserver.cc:460] Webserver started at http://127.1.245.66:36181/ using document root <none> and password file <none>
I20260806 18:44:44.582635 4758 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:44.582711 4758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:44.582913 4758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:44.585132 4758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "56b64d8999cb4689afb25fa8a4897162"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "f3b8d0a0e9e0604e0d87a7e493c767ba"
server_key_iv: "5e312114582dd9fc5d63858058363633"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.585692 4758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "56b64d8999cb4689afb25fa8a4897162"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "f3b8d0a0e9e0604e0d87a7e493c767ba"
server_key_iv: "5e312114582dd9fc5d63858058363633"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.589486 4758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:44:44.592110 4774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:44.593459 4758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:44:44.593587 4758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "56b64d8999cb4689afb25fa8a4897162"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "f3b8d0a0e9e0604e0d87a7e493c767ba"
server_key_iv: "5e312114582dd9fc5d63858058363633"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.593695 4758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:44.610867 4758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:44.614091 4758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:44.614336 4758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:44.615883 4758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:44.615937 4758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:44.615991 4758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:44.616021 4758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:44.626381 4758 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:45613
I20260806 18:44:44.626448 4939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:45613 every 8 connection(s)
I20260806 18:44:44.627494 4758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:44:44.630352 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.616819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:36765 (local address 127.1.245.126:45825)
0806 18:44:44.616951 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.616955 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.618031 (+ 1076us) server_negotiation.cc:407] Connection header received
0806 18:44:44.619161 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.619165 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.619231 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.619307 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:44.621037 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.621875 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.623341 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.623747 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.623847 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.626546 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:44.626552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:44.626556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:44.626586 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:44.628206 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.628785 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.628788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.628789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.628867 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.629203 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.629207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.629208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.629431 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:44.629517 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.630029 (+ 512us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.630150 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:44:44.631105 4784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.617347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45825 (local address 127.1.245.66:36765)
0806 18:44:44.617858 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.617901 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.618899 (+ 998us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.619756 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.619764 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.620168 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:44.620853 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.620874 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.621982 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.621986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.623194 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.623207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.623908 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.623912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.624081 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.624688 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:44.624706 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:44.626307 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:44.628328 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.628333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.628348 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.628669 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.628958 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.628961 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.628963 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.629097 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.629541 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:44.629547 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:44.629785 (+ 238us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.629942 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.630257 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":124,"threads_started":1}
I20260806 18:44:44.631927 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 4758
I20260806 18:44:44.632068 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:44:44.632432 2005 external_mini_cluster.cc:1428] Setting key d992fa8ac3ca4a6427ad8dceb9ed4d90
I20260806 18:44:44.632465 4941 heartbeater.cc:347] Connected to a master server at 127.1.245.126:45825
I20260806 18:44:44.632606 4941 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:44.633157 4941 heartbeater.cc:511] Master 127.1.245.126:45825 requested a full tablet report, sending...
I20260806 18:44:44.634240 4450 ts_manager.cc:194] Registered new tserver with Master: 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613)
I20260806 18:44:44.634944 4450 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:36765
I20260806 18:44:44.642689 4951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.634811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45825 (local address 127.1.245.66:51423)
0806 18:44:44.635114 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.635128 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.635224 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.635494 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.635497 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.635709 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:44.635850 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.635858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.636926 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.636929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.637730 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.637741 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.638380 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.638384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.638466 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.638959 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:44.638971 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:44.639283 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:44.641116 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.641119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.641120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.641330 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.641604 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.641606 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.641608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.641677 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.642151 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:44.642154 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:44.642236 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.642361 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.642537 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":87,"threads_started":1}
I20260806 18:44:44.642704 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.634858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:51423 (local address 127.1.245.126:45825)
0806 18:44:44.634957 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.634961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.635185 (+ 224us) server_negotiation.cc:407] Connection header received
0806 18:44:44.635314 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.635317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.635378 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.635459 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:44.635985 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.636739 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.637901 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.638228 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.638351 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.639443 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:44.639448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:44.639450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:44.639479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:44.640984 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.641430 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.641434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.641436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.641524 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.641764 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.641767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.641768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.642052 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:44.642152 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.642427 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.642545 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:44:44.665645 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45825
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:44.780652 4954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:44.780937 4954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:44.780988 4954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:44.785187 4954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:44.785265 4954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:44.785306 4954 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 18:44:44.785319 4954 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:44:44.785352 4954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:44:44.790874 4954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:45825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:44.792026 4954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:44.793171 4954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:44:44.803642 4954 server_base.cc:1034] running on GCE node
W20260806 18:44:44.803587 4961 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:44.803594 4960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:44.803606 4963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:44.804433 4954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:44.805092 4954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:44.806284 4954 hybrid_clock.cc:648] HybridClock initialized: now 1786041884806282 us; error 38 us; skew 500 ppm
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:44.809144 4954 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:44:44.810114 4954 webserver.cc:460] Webserver started at http://127.1.245.67:44711/ using document root <none> and password file <none>
I20260806 18:44:44.810719 4954 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:44.810789 4954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:44.810977 4954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:44.812983 4954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "f0d928d8198046f68f10864efd913d7f"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "a263ee402ccd69ca5312b67ca6124e28"
server_key_iv: "934c0682930561fe4d4bf57a66ec06d2"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.813517 4954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "f0d928d8198046f68f10864efd913d7f"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "a263ee402ccd69ca5312b67ca6124e28"
server_key_iv: "934c0682930561fe4d4bf57a66ec06d2"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.817397 4954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:44:44.819993 4972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:44.821270 4954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:44:44.821430 4954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "f0d928d8198046f68f10864efd913d7f"
format_stamp: "Formatted at 2026-08-06 18:44:44 on dist-test-slave-fgb3"
server_key: "a263ee402ccd69ca5312b67ca6124e28"
server_key_iv: "934c0682930561fe4d4bf57a66ec06d2"
server_key_version: "encryptionkey@0"
I20260806 18:44:44.821542 4954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:44.833168 4954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:44.836436 4954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:44.836591 4954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:44.837996 4954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:44.838050 4954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:44.838106 4954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:44.838137 4954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:44.847708 4954 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:39961
I20260806 18:44:44.847910 5135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:39961 every 8 connection(s)
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:44.848624 4954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:44:44.852146 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.838814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:50421 (local address 127.1.245.126:45825)
0806 18:44:44.838981 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.838986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.839842 (+ 856us) server_negotiation.cc:407] Connection header received
0806 18:44:44.840897 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.840901 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.840965 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.841063 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:44.842777 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.843617 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.845300 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.845546 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.845661 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.848580 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:44.848583 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:44.848586 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:44.848612 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:44.850067 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.850628 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.850632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.850634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.850717 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.851082 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.851085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.851087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.851272 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:44.851366 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.851864 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.851987 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:44:44.852551 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 4954
I20260806 18:44:44.852667 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:44:44.852964 2005 external_mini_cluster.cc:1428] Setting key 8849c46a06e743e079389c568c386402
I20260806 18:44:44.853068 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.839201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45825 (local address 127.1.245.67:50421)
0806 18:44:44.839669 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.839718 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.840645 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.841380 (+ 735us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.841388 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.841871 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:44.842625 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.842649 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.843780 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.843785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.845171 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.845183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.845734 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.845739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.845956 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.846667 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:44.846691 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:44.848407 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:44.850183 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.850189 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.850203 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.850522 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.850809 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.850812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.850814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.850973 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.851380 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:44.851386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:44.851621 (+ 235us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.851776 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.852094 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":116,"threads_started":1}
I20260806 18:44:44.854269 5137 heartbeater.cc:347] Connected to a master server at 127.1.245.126:45825
I20260806 18:44:44.854441 5137 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:44.854918 5137 heartbeater.cc:511] Master 127.1.245.126:45825 requested a full tablet report, sending...
I20260806 18:44:44.855885 4451 ts_manager.cc:194] Registered new tserver with Master: f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961)
I20260806 18:44:44.856447 4451 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:50421
I20260806 18:44:44.856865 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:44:44.863660 5147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.856819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45825 (local address 127.1.245.67:44597)
0806 18:44:44.857130 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.857143 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.857241 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.857525 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.857528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.857709 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:44.857800 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.857805 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.858612 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.858615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.859160 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.859168 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.859607 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.859611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.859698 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.860180 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:44.860190 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:44.860484 (+ 294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:44.862014 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.862018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.862020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.862271 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.862599 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:44.862602 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:44.862604 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:44.862699 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:44.863128 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:44.863131 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:44.863225 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.863330 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.863471 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":91,"threads_started":1}
I20260806 18:44:44.863688 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.856968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:44597 (local address 127.1.245.126:45825)
0806 18:44:44.857062 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.857066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.857222 (+ 156us) server_negotiation.cc:407] Connection header received
0806 18:44:44.857312 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.857316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.857356 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.857418 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:44.857917 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.858493 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.859277 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.859501 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.859585 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.860618 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:44.860622 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:44.860625 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:44.860649 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:44.861904 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.862418 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.862421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.862423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.862502 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.862839 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.862842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.862843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.863021 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:44.863109 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.863411 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.863536 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:44:44 dist-test-slave-fgb3 krb5kdc[4386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:44.886656 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.876311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50394 (local address 127.1.245.126:45825)
0806 18:44:44.876469 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.876473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.876486 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:44:44.876526 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.876529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.876568 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.876634 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:44.877527 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.878400 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.879599 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.879863 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.879921 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.883626 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:44.883630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:44.883633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:44.883670 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:44.885107 (+ 1437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.885604 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.885607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.885608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.885691 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:44.885927 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:44.885928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:44.885930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:44.886098 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:44.886191 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.886408 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.886520 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:44:44.889626 4451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:44:44.892695 4451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:44.907577 5161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.901534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34183 (local address 127.0.0.1:58700)
0806 18:44:44.901789 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.901805 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.901894 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.903707 (+ 1813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.903709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.903726 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:44.903830 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.903837 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.905129 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.905133 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.906108 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.906118 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.907018 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.907021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.907140 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.907179 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.907288 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.907413 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":42,"thread_start_us":102,"threads_started":1}
I20260806 18:44:44.907730 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.901354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:39961 (local address 127.0.0.1:35954)
0806 18:44:44.901695 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.901744 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.901887 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.903175 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.903178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.903197 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:44.903343 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.903351 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.904674 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.904677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.906139 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.906146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.907231 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.907235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.907411 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.907433 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.907504 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.907565 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":131,"thread_start_us":75,"threads_started":1}
I20260806 18:44:44.908049 5162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.902073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:45613 (local address 127.0.0.1:46254)
0806 18:44:44.902293 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:44.902333 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:44.902415 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:44.903125 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:44.903131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:44.903157 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:44.903472 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.903479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.904918 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.904921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.906118 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:44.906129 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.907226 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.907229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:44.907376 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.907418 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 18:44:44.907472 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 18:44:44.907542 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":212,"thread_start_us":85,"threads_started":1}
I20260806 18:44:44.908041 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.902924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.1.245.65:34183)
0806 18:44:44.903298 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.903303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.903324 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:44:44.903392 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.903397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.903569 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.903718 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:44.903957 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.905019 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.906247 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.906890 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.906994 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.907150 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.907786 (+ 636us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.907888 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":87,"threads_started":1}
I20260806 18:44:44.908053 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.902277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46254 (local address 127.1.245.66:45613)
0806 18:44:44.902635 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.902645 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.902665 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:44:44.902729 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.902733 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.902933 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.903330 (+ 397us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:44.903588 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.904799 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.906259 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.907086 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.907625 (+ 539us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.907780 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.907832 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.907917 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":164,"threads_started":1}
I20260806 18:44:44.908723 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:44.902400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35954 (local address 127.1.245.67:39961)
0806 18:44:44.902724 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:44.902737 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:44.902758 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:44:44.902834 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:44.902839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:44.903027 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:44.903634 (+ 607us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:44.903777 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.904559 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.906259 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:44.907086 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:44.907231 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:44.907394 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:44.907552 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:44:44.907649 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":119,"threads_started":1}
I20260806 18:44:44.911002 4651 tablet_service.cc:1467] Processing CreateTablet for tablet e874effce54a4facba119a29130e2f56 (DEFAULT_TABLE table=test-table [id=d88e88141b0f45139f586b14f2e4b7e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:44.911063 4843 tablet_service.cc:1467] Processing CreateTablet for tablet e874effce54a4facba119a29130e2f56 (DEFAULT_TABLE table=test-table [id=d88e88141b0f45139f586b14f2e4b7e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:44.911362 5039 tablet_service.cc:1467] Processing CreateTablet for tablet e874effce54a4facba119a29130e2f56 (DEFAULT_TABLE table=test-table [id=d88e88141b0f45139f586b14f2e4b7e3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:44.912026 4651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e874effce54a4facba119a29130e2f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:44.912088 4843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e874effce54a4facba119a29130e2f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:44.912204 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e874effce54a4facba119a29130e2f56. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:44.918133 5169 tablet_bootstrap.cc:492] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Bootstrap starting.
I20260806 18:44:44.920738 5171 tablet_bootstrap.cc:492] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Bootstrap starting.
I20260806 18:44:44.920975 5169 tablet_bootstrap.cc:654] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:44.921690 5169 log.cc:824] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:44.922278 5170 tablet_bootstrap.cc:492] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Bootstrap starting.
I20260806 18:44:44.922690 5171 tablet_bootstrap.cc:654] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:44.923524 5171 log.cc:824] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:44.923796 5169 tablet_bootstrap.cc:492] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: No bootstrap required, opened a new log
I20260806 18:44:44.924003 5169 ts_tablet_manager.cc:1397] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260806 18:44:44.924197 5170 tablet_bootstrap.cc:654] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:44.925027 5170 log.cc:824] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:44.925573 5171 tablet_bootstrap.cc:492] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: No bootstrap required, opened a new log
I20260806 18:44:44.925774 5171 ts_tablet_manager.cc:1397] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:44:44.926785 5170 tablet_bootstrap.cc:492] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: No bootstrap required, opened a new log
I20260806 18:44:44.926970 5170 ts_tablet_manager.cc:1397] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260806 18:44:44.929095 5169 raft_consensus.cc:348] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:44.929394 5169 raft_consensus.cc:374] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:44.929452 5169 raft_consensus.cc:729] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52be427786d247159029b3cdf492b0c0, State: Initialized, Role: FOLLOWER
I20260806 18:44:44.929518 5171 raft_consensus.cc:348] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:44.929744 5171 raft_consensus.cc:374] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:44.929809 5171 raft_consensus.cc:729] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b64d8999cb4689afb25fa8a4897162, State: Initialized, Role: FOLLOWER
I20260806 18:44:44.929930 5169 consensus_queue.cc:260] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:44.929906 5170 raft_consensus.cc:348] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:44.930143 5170 raft_consensus.cc:374] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:44.930212 5170 raft_consensus.cc:729] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d928d8198046f68f10864efd913d7f, State: Initialized, Role: FOLLOWER
I20260806 18:44:44.930203 5171 consensus_queue.cc:260] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:44.930636 5170 consensus_queue.cc:260] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:44.931041 5169 ts_tablet_manager.cc:1428] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260806 18:44:44.931046 4941 heartbeater.cc:503] Master 127.1.245.126:45825 was elected leader, sending a full tablet report...
I20260806 18:44:44.931111 5171 ts_tablet_manager.cc:1428] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260806 18:44:44.931393 4744 heartbeater.cc:503] Master 127.1.245.126:45825 was elected leader, sending a full tablet report...
I20260806 18:44:44.931735 5137 heartbeater.cc:503] Master 127.1.245.126:45825 was elected leader, sending a full tablet report...
I20260806 18:44:44.932435 5170 ts_tablet_manager.cc:1428] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:44:45.052950 5178 raft_consensus.cc:482] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:45.053174 5178 raft_consensus.cc:504] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:45.054471 5178 leader_election.cc:288] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961), 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183)
I20260806 18:44:45.058763 4784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.054706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:39961 (local address 127.1.245.66:38831)
0806 18:44:45.054847 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.054862 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.054960 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.055285 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.055288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.055333 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:45.055462 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.055469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.056372 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.056375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.057213 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.057223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.058211 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.058214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.058436 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.058465 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.058542 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.058619 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"spinlock_wait_cycles":29440}
I20260806 18:44:45.058759 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.054913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:38831 (local address 127.1.245.67:39961)
0806 18:44:45.055068 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.055072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.055087 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:44:45.055140 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.055143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.055189 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.055265 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:45.055594 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.056246 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.057373 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.058087 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.058228 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.058294 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.058570 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.058642 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:44:45.059449 5076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e874effce54a4facba119a29130e2f56" candidate_uuid: "56b64d8999cb4689afb25fa8a4897162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d928d8198046f68f10864efd913d7f" is_pre_election: true
I20260806 18:44:45.059561 5182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.054726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34183 (local address 127.1.245.66:40309)
0806 18:44:45.055143 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.055157 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.055341 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.055652 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.055654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.055676 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:45.055788 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.055794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.056586 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.056589 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.058344 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.058357 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.059188 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.059193 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.059290 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.059311 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.059390 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.059443 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"spinlock_wait_cycles":5504,"thread_start_us":149,"threads_started":1}
I20260806 18:44:45.059610 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.054915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:40309 (local address 127.1.245.65:34183)
0806 18:44:45.055121 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.055125 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.055249 (+ 124us) server_negotiation.cc:407] Connection header received
0806 18:44:45.055486 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.055489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.055539 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.055619 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:45.055920 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.056443 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.058493 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.059073 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.059178 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.059225 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.059423 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.059499 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260806 18:44:45.059814 5076 raft_consensus.cc:2454] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b64d8999cb4689afb25fa8a4897162 in term 0.
I20260806 18:44:45.060233 4678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e874effce54a4facba119a29130e2f56" candidate_uuid: "56b64d8999cb4689afb25fa8a4897162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0" is_pre_election: true
I20260806 18:44:45.060362 4776 leader_election.cc:302] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56b64d8999cb4689afb25fa8a4897162, f0d928d8198046f68f10864efd913d7f; no voters:
I20260806 18:44:45.060555 4678 raft_consensus.cc:2454] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b64d8999cb4689afb25fa8a4897162 in term 0.
I20260806 18:44:45.060645 5178 raft_consensus.cc:2781] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:45.060737 5178 raft_consensus.cc:482] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:45.060786 5178 raft_consensus.cc:3037] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.061857 5178 raft_consensus.cc:504] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:45.062207 5178 leader_election.cc:288] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [CANDIDATE]: Term 1 election: Requested vote from peers f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961), 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183)
I20260806 18:44:45.062647 4678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e874effce54a4facba119a29130e2f56" candidate_uuid: "56b64d8999cb4689afb25fa8a4897162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0"
I20260806 18:44:45.062623 5076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e874effce54a4facba119a29130e2f56" candidate_uuid: "56b64d8999cb4689afb25fa8a4897162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d928d8198046f68f10864efd913d7f"
I20260806 18:44:45.062775 4678 raft_consensus.cc:3037] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.062774 5076 raft_consensus.cc:3037] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.064114 5076 raft_consensus.cc:2454] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b64d8999cb4689afb25fa8a4897162 in term 1.
I20260806 18:44:45.064114 4678 raft_consensus.cc:2454] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b64d8999cb4689afb25fa8a4897162 in term 1.
I20260806 18:44:45.064440 4778 leader_election.cc:302] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52be427786d247159029b3cdf492b0c0, 56b64d8999cb4689afb25fa8a4897162; no voters:
I20260806 18:44:45.064621 5178 raft_consensus.cc:2781] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:45.064862 5178 raft_consensus.cc:686] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 1 LEADER]: Becoming Leader. State: Replica: 56b64d8999cb4689afb25fa8a4897162, State: Running, Role: LEADER
I20260806 18:44:45.065191 5178 consensus_queue.cc:237] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } }
I20260806 18:44:45.068490 4449 catalog_manager.cc:5495] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "56b64d8999cb4689afb25fa8a4897162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } health_report { overall_health: HEALTHY } } }
W20260806 18:44:45.100148 5139 tablet.cc:2369] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 18:44:45 dist-test-slave-fgb3 krb5kdc[4386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041884, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
W20260806 18:44:45.128803 4943 tablet.cc:2369] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:44:45.129158 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.120426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45825 (local address 127.0.0.1:50402)
0806 18:44:45.120755 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.120768 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.120895 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.121152 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.121155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.121586 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:45.121686 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.121693 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.122514 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.122517 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.123057 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.123065 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.123491 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.123495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.123587 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.124151 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:45.124164 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:45.125721 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:45.127575 (+ 1854us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:45.127586 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:45.127594 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:45.127829 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:45.128105 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:45.128108 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:45.128110 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:45.128187 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:45.128593 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:45.128601 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:45.128714 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.128815 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.128983 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":121,"threads_started":1}
I20260806 18:44:45.129287 4520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.120543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.1.245.126:45825)
0806 18:44:45.120687 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.120692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.120859 (+ 167us) server_negotiation.cc:407] Connection header received
0806 18:44:45.120992 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.120995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.121033 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.121098 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:45.121825 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.122398 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.123174 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.123380 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.123467 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.125855 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:45.125858 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:45.125860 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:45.125885 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:45.127450 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:45.127932 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:45.127936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:45.127938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:45.128014 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:45.128275 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:45.128278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:45.128280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:45.128460 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:45.128547 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.128887 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.129138 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":133,"server-negotiator.queue_time_us":59}
I20260806 18:44:45.133730 4449 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:50402:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:44:45.134527 4449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:45.139775 4651 tablet_service.cc:1467] Processing CreateTablet for tablet daf1b97cb36a487f9d95bc3baba0e426 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=783c3a5afc524d10a32cce19f3fb4cf0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:44:45.139811 4843 tablet_service.cc:1467] Processing CreateTablet for tablet daf1b97cb36a487f9d95bc3baba0e426 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=783c3a5afc524d10a32cce19f3fb4cf0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:44:45.140092 5039 tablet_service.cc:1467] Processing CreateTablet for tablet daf1b97cb36a487f9d95bc3baba0e426 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=783c3a5afc524d10a32cce19f3fb4cf0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:44:45.140496 4651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daf1b97cb36a487f9d95bc3baba0e426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:45.140543 4843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daf1b97cb36a487f9d95bc3baba0e426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:45.140609 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet daf1b97cb36a487f9d95bc3baba0e426. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:45.143697 5169 tablet_bootstrap.cc:492] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: Bootstrap starting.
I20260806 18:44:45.143862 5170 tablet_bootstrap.cc:492] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: Bootstrap starting.
I20260806 18:44:45.143919 5171 tablet_bootstrap.cc:492] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: Bootstrap starting.
I20260806 18:44:45.144752 5171 tablet_bootstrap.cc:654] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:45.144842 5170 tablet_bootstrap.cc:654] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:45.144842 5169 tablet_bootstrap.cc:654] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:45.145982 5171 tablet_bootstrap.cc:492] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: No bootstrap required, opened a new log
I20260806 18:44:45.146059 5171 ts_tablet_manager.cc:1397] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
W20260806 18:44:45.146552 4745 tablet.cc:2369] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:44:45.146534 5171 raft_consensus.cc:348] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.146647 5171 raft_consensus.cc:374] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:45.146689 5171 raft_consensus.cc:729] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b64d8999cb4689afb25fa8a4897162, State: Initialized, Role: FOLLOWER
I20260806 18:44:45.146819 5171 consensus_queue.cc:260] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.147076 5171 ts_tablet_manager.cc:1428] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:45.147096 5178 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.147222 5178 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:45.148012 5169 tablet_bootstrap.cc:492] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: No bootstrap required, opened a new log
I20260806 18:44:45.148095 5169 ts_tablet_manager.cc:1397] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 18:44:45.148386 5170 tablet_bootstrap.cc:492] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: No bootstrap required, opened a new log
I20260806 18:44:45.148497 5170 ts_tablet_manager.cc:1397] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260806 18:44:45.148537 5169 raft_consensus.cc:348] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.148648 5169 raft_consensus.cc:374] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:45.148687 5169 raft_consensus.cc:729] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52be427786d247159029b3cdf492b0c0, State: Initialized, Role: FOLLOWER
I20260806 18:44:45.148823 5169 consensus_queue.cc:260] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.148976 5170 raft_consensus.cc:348] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.149089 5170 raft_consensus.cc:374] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:45.149122 5170 raft_consensus.cc:729] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d928d8198046f68f10864efd913d7f, State: Initialized, Role: FOLLOWER
I20260806 18:44:45.149106 5177 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.149251 5177 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:45.149482 5169 ts_tablet_manager.cc:1428] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:45.149796 5170 consensus_queue.cc:260] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.150077 5180 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.150225 5180 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:45.150509 5170 ts_tablet_manager.cc:1428] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 18:44:45.338558 5180 raft_consensus.cc:482] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:45.338733 5180 raft_consensus.cc:504] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.339931 5180 leader_election.cc:288] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183), 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613)
I20260806 18:44:45.344035 5168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.340386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:38827 (local address 127.1.245.65:34183)
0806 18:44:45.340536 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.340540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.340554 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:44:45.340605 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.340608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.340648 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.340731 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:45.341041 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.341684 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.342914 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.343529 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.343638 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.343694 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.343855 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.343919 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:44:45.344054 4983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.340208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34183 (local address 127.1.245.67:38827)
0806 18:44:45.340325 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.340341 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.340443 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.340782 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.340785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.340808 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:45.340923 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.340930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.341807 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.341810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.342784 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.342793 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.343639 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.343642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.343738 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.343758 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.343828 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.343897 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 18:44:45.344597 5195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.340494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:45613 (local address 127.1.245.67:60021)
0806 18:44:45.340789 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.340802 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.340895 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.341247 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.341250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.341269 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:44:45.341394 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.341399 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.342294 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.342297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.343281 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.343291 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.344043 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.344047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.344305 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.344331 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.344399 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.344464 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":103,"threads_started":1}
I20260806 18:44:45.344635 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.340893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:60021 (local address 127.1.245.66:45613)
0806 18:44:45.341046 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.341050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.341063 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:44:45.341110 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.341112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.341149 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.341206 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:44:45.341492 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.342155 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.343433 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.343907 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.344018 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.344067 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.344439 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.344517 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:44:45.344651 4678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daf1b97cb36a487f9d95bc3baba0e426" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0" is_pre_election: true
I20260806 18:44:45.344803 4678 raft_consensus.cc:2454] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 0.
I20260806 18:44:45.345139 4977 leader_election.cc:302] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52be427786d247159029b3cdf492b0c0, f0d928d8198046f68f10864efd913d7f; no voters:
I20260806 18:44:45.345402 5180 raft_consensus.cc:2781] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:45.345475 5180 raft_consensus.cc:482] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:45.345526 5180 raft_consensus.cc:3037] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.345810 4873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daf1b97cb36a487f9d95bc3baba0e426" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b64d8999cb4689afb25fa8a4897162" is_pre_election: true
I20260806 18:44:45.346009 4873 raft_consensus.cc:2454] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 0.
I20260806 18:44:45.346680 5180 raft_consensus.cc:504] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.347004 5180 leader_election.cc:288] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 election: Requested vote from peers 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183), 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613)
I20260806 18:44:45.347309 4678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daf1b97cb36a487f9d95bc3baba0e426" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0"
I20260806 18:44:45.347364 4873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "daf1b97cb36a487f9d95bc3baba0e426" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b64d8999cb4689afb25fa8a4897162"
I20260806 18:44:45.347434 4678 raft_consensus.cc:3037] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.347455 4873 raft_consensus.cc:3037] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.348519 4873 raft_consensus.cc:2454] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 1.
I20260806 18:44:45.348519 4678 raft_consensus.cc:2454] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 1.
I20260806 18:44:45.348781 4977 leader_election.cc:302] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56b64d8999cb4689afb25fa8a4897162, f0d928d8198046f68f10864efd913d7f; no voters:
I20260806 18:44:45.348940 5180 raft_consensus.cc:2781] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:45.349129 5180 raft_consensus.cc:686] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [term 1 LEADER]: Becoming Leader. State: Replica: f0d928d8198046f68f10864efd913d7f, State: Running, Role: LEADER
I20260806 18:44:45.349455 5180 consensus_queue.cc:237] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.351778 5196 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: TxnStatusTablet state changed. Reason: New leader f0d928d8198046f68f10864efd913d7f. Latest consensus state: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.351895 5196 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:44:45.352176 5198 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:44:45.352391 5198 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:44:45.352850 4451 catalog_manager.cc:5495] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f reported cstate change: term changed from 0 to 1, leader changed from <none> to f0d928d8198046f68f10864efd913d7f (127.1.245.67). New cstate: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } health_report { overall_health: HEALTHY } } }
I20260806 18:44:45.386823 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.382595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35958 (local address 127.1.245.67:39961)
0806 18:44:45.382725 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.382728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.382742 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:44:45.382938 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.382941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.383007 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.383104 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:45.383434 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.383961 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.385187 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.385413 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.385497 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.385888 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.386402 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.386504 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.386615 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.386668 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:44:45.386866 5191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.382493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:39961 (local address 127.0.0.1:35958)
0806 18:44:45.382636 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.382650 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.382786 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.383146 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.383149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.383158 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:44:45.383323 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.383331 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.384074 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.384077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.385058 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.385071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.385533 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.385537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.385643 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.385702 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.386509 (+ 807us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.386525 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.386597 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.386710 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260806 18:44:45.392316 4873 raft_consensus.cc:1261] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Refusing update from remote peer f0d928d8198046f68f10864efd913d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:45.392411 4678 raft_consensus.cc:1261] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Refusing update from remote peer f0d928d8198046f68f10864efd913d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:45.393000 5180 consensus_queue.cc:1035] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:45.393210 5196 consensus_queue.cc:1035] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:45.396509 5178 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: TxnStatusTablet state changed. Reason: New leader f0d928d8198046f68f10864efd913d7f. Latest consensus state: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.396638 5178 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:45.398602 5177 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: TxnStatusTablet state changed. Reason: New leader f0d928d8198046f68f10864efd913d7f. Latest consensus state: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.398742 5177 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:45.399654 5180 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.399808 5180 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:44:45.399958 5178 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.400058 5178 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P 56b64d8999cb4689afb25fa8a4897162: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:45.400902 5203 mvcc.cc:205] Tried to move back new op lower bound from 7315627562559639552 to 7315627562393890816. Current Snapshot: MvccSnapshot[applied={T|T < 7315627562559639552}]
I20260806 18:44:45.404218 5207 mvcc.cc:205] Tried to move back new op lower bound from 7315627562559639552 to 7315627562393890816. Current Snapshot: MvccSnapshot[applied={T|T < 7315627562559639552}]
I20260806 18:44:45.406387 5180 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.406512 5180 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P f0d928d8198046f68f10864efd913d7f: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:44:45.408102 5177 tablet_replica.cc:439] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } } }
I20260806 18:44:45.408236 5177 tablet_replica.cc:442] T daf1b97cb36a487f9d95bc3baba0e426 P 52be427786d247159029b3cdf492b0c0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:44:45.410363 5209 mvcc.cc:205] Tried to move back new op lower bound from 7315627562559639552 to 7315627562393890816. Current Snapshot: MvccSnapshot[applied={T|T < 7315627562559639552}]
I20260806 18:44:45.415158 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.411206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46266 (local address 127.1.245.66:45613)
0806 18:44:45.411339 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.411344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.411553 (+ 209us) server_negotiation.cc:407] Connection header received
0806 18:44:45.411708 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.411712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.411773 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.411865 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:45.412159 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.412680 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.413620 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.413849 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.413956 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.414263 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.414736 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.414851 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.414951 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.415013 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:44:45.427609 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.423684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:35801 (local address 127.1.245.67:39961)
0806 18:44:45.423865 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.423868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.423961 (+ 93us) server_negotiation.cc:407] Connection header received
0806 18:44:45.424135 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.424139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.424199 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.424282 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:45.424549 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.425164 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.426425 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.426671 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.426806 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.427120 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.427230 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.427299 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.427445 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.427495 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260806 18:44:45.427685 5221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.423535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:39961 (local address 127.1.245.66:35801)
0806 18:44:45.423873 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.423888 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.423998 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.424303 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.424306 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.424314 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:44:45.424446 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.424452 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.425276 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.425279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.426282 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.426294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.426800 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.426803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.426900 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.426957 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.427339 (+ 382us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.427347 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.427430 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.427511 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":119,"threads_started":1}
I20260806 18:44:45.435266 4678 raft_consensus.cc:1261] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Refusing update from remote peer 56b64d8999cb4689afb25fa8a4897162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:45.435573 5076 raft_consensus.cc:1261] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Refusing update from remote peer 56b64d8999cb4689afb25fa8a4897162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:45.435777 5183 consensus_queue.cc:1035] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:45.436208 5183 consensus_queue.cc:1035] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:45.460865 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.456125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:49095 (local address 127.1.245.66:45613)
0806 18:44:45.456247 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.456251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.456892 (+ 641us) server_negotiation.cc:407] Connection header received
0806 18:44:45.457001 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.457003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.457049 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.457109 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:45.457364 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.457862 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.459473 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.459832 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.459912 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.460427 (+ 515us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.460506 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.460562 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.460682 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.460735 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:44:45.460923 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.456071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:45613 (local address 127.1.245.67:49095)
0806 18:44:45.456794 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.456810 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.456921 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.457145 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.457148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.457156 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:44:45.457267 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.457273 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.457995 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.457998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.459329 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.459342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.460150 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.460153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.460242 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.460299 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.460596 (+ 297us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.460603 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.460676 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.460772 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":104,"threads_started":1}
I20260806 18:44:45.487504 4451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:45.487721 4451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:45.490833 4451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: Sending DeleteTablet for 3 replicas of tablet e874effce54a4facba119a29130e2f56
I20260806 18:44:45.491905 4843 tablet_service.cc:1514] Processing DeleteTablet for tablet e874effce54a4facba119a29130e2f56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:45 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:46254
I20260806 18:44:45.491950 4651 tablet_service.cc:1514] Processing DeleteTablet for tablet e874effce54a4facba119a29130e2f56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:45 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:58700
I20260806 18:44:45.492697 5039 tablet_service.cc:1514] Processing DeleteTablet for tablet e874effce54a4facba119a29130e2f56 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:45 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35954
I20260806 18:44:45.492867 4451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:44:45.493304 5229 tablet_replica.cc:330] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: stopping tablet replica
W20260806 18:44:45.493321 4451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:45.493674 5229 raft_consensus.cc:2229] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:45.493938 5229 raft_consensus.cc:2258] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:45.494526 5234 tablet_replica.cc:330] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: stopping tablet replica
I20260806 18:44:45.494851 5234 raft_consensus.cc:2229] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:45.495025 5234 raft_consensus.cc:2258] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:45.497429 5229 ts_tablet_manager.cc:1905] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:45.497610 5234 ts_tablet_manager.cc:1905] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:45.499476 4651 tablet_service.cc:1467] Processing CreateTablet for tablet 144e0f9bf6b747e9a2bcf79738f00008 (DEFAULT_TABLE table=test-table [id=e6aee0d8655545a2a64407037819a04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:45.499464 4843 tablet_service.cc:1467] Processing CreateTablet for tablet 144e0f9bf6b747e9a2bcf79738f00008 (DEFAULT_TABLE table=test-table [id=e6aee0d8655545a2a64407037819a04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:45.499754 4651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144e0f9bf6b747e9a2bcf79738f00008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:45.499755 4843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144e0f9bf6b747e9a2bcf79738f00008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:45.499927 5230 tablet_replica.cc:330] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: stopping tablet replica
I20260806 18:44:45.500203 5230 raft_consensus.cc:2229] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:44:45.500593 5230 raft_consensus.cc:2258] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:45.500924 5234 ts_tablet_manager.cc:1918] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:45.501014 5234 log.cc:1196] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/e874effce54a4facba119a29130e2f56
I20260806 18:44:45.501327 5234 ts_tablet_manager.cc:1939] T e874effce54a4facba119a29130e2f56 P f0d928d8198046f68f10864efd913d7f: Deleting consensus metadata
I20260806 18:44:45.501871 5230 ts_tablet_manager.cc:1905] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:45.502141 5169 tablet_bootstrap.cc:492] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: Bootstrap starting.
I20260806 18:44:45.502213 5171 tablet_bootstrap.cc:492] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: Bootstrap starting.
I20260806 18:44:45.503090 5171 tablet_bootstrap.cc:654] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:45.503090 5169 tablet_bootstrap.cc:654] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:45.504240 4427 catalog_manager.cc:4841] TS f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961): tablet e874effce54a4facba119a29130e2f56 (table test-table [id=d88e88141b0f45139f586b14f2e4b7e3]) successfully deleted
I20260806 18:44:45.504542 5230 ts_tablet_manager.cc:1918] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:45.504637 5230 log.cc:1196] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/e874effce54a4facba119a29130e2f56
I20260806 18:44:45.504822 5039 tablet_service.cc:1467] Processing CreateTablet for tablet 144e0f9bf6b747e9a2bcf79738f00008 (DEFAULT_TABLE table=test-table [id=e6aee0d8655545a2a64407037819a04e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:45.504932 5230 ts_tablet_manager.cc:1939] T e874effce54a4facba119a29130e2f56 P 56b64d8999cb4689afb25fa8a4897162: Deleting consensus metadata
I20260806 18:44:45.504978 5229 ts_tablet_manager.cc:1918] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:45.505056 5229 log.cc:1196] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/e874effce54a4facba119a29130e2f56
I20260806 18:44:45.505065 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 144e0f9bf6b747e9a2bcf79738f00008. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:45.505345 5229 ts_tablet_manager.cc:1939] T e874effce54a4facba119a29130e2f56 P 52be427786d247159029b3cdf492b0c0: Deleting consensus metadata
I20260806 18:44:45.506043 4431 catalog_manager.cc:4841] TS 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613): tablet e874effce54a4facba119a29130e2f56 (table test-table [id=d88e88141b0f45139f586b14f2e4b7e3]) successfully deleted
I20260806 18:44:45.506474 4431 catalog_manager.cc:4841] TS 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183): tablet e874effce54a4facba119a29130e2f56 (table test-table [id=d88e88141b0f45139f586b14f2e4b7e3]) successfully deleted
I20260806 18:44:45.507272 5170 tablet_bootstrap.cc:492] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: Bootstrap starting.
I20260806 18:44:45.508162 5170 tablet_bootstrap.cc:654] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:45.509553 5170 tablet_bootstrap.cc:492] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: No bootstrap required, opened a new log
I20260806 18:44:45.509646 5170 ts_tablet_manager.cc:1397] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 18:44:45.510093 5170 raft_consensus.cc:348] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.510216 5170 raft_consensus.cc:374] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:45.510249 5170 raft_consensus.cc:729] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d928d8198046f68f10864efd913d7f, State: Initialized, Role: FOLLOWER
I20260806 18:44:45.510401 5170 consensus_queue.cc:260] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.510668 5170 ts_tablet_manager.cc:1428] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:45.511003 5169 tablet_bootstrap.cc:492] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: No bootstrap required, opened a new log
I20260806 18:44:45.511083 5169 ts_tablet_manager.cc:1397] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260806 18:44:45.511268 5171 tablet_bootstrap.cc:492] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: No bootstrap required, opened a new log
I20260806 18:44:45.511345 5171 ts_tablet_manager.cc:1397] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260806 18:44:45.511543 5169 raft_consensus.cc:348] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.511646 5169 raft_consensus.cc:374] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:45.511677 5169 raft_consensus.cc:729] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52be427786d247159029b3cdf492b0c0, State: Initialized, Role: FOLLOWER
I20260806 18:44:45.511765 5171 raft_consensus.cc:348] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.511797 5169 consensus_queue.cc:260] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.511868 5171 raft_consensus.cc:374] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:45.511896 5171 raft_consensus.cc:729] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b64d8999cb4689afb25fa8a4897162, State: Initialized, Role: FOLLOWER
I20260806 18:44:45.512004 5171 consensus_queue.cc:260] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.512055 5169 ts_tablet_manager.cc:1428] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:45.512264 5171 ts_tablet_manager.cc:1428] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:45.691566 5196 raft_consensus.cc:482] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:45.691753 5196 raft_consensus.cc:504] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.692255 5196 leader_election.cc:288] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183), 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613)
I20260806 18:44:45.692871 4678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144e0f9bf6b747e9a2bcf79738f00008" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0" is_pre_election: true
I20260806 18:44:45.692907 4873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144e0f9bf6b747e9a2bcf79738f00008" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b64d8999cb4689afb25fa8a4897162" is_pre_election: true
I20260806 18:44:45.693040 4873 raft_consensus.cc:2454] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 0.
I20260806 18:44:45.693033 4678 raft_consensus.cc:2454] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 0.
I20260806 18:44:45.693363 4977 leader_election.cc:302] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56b64d8999cb4689afb25fa8a4897162, f0d928d8198046f68f10864efd913d7f; no voters:
I20260806 18:44:45.693557 5196 raft_consensus.cc:2781] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:45.693621 5196 raft_consensus.cc:482] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:45.693666 5196 raft_consensus.cc:3037] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.694790 5196 raft_consensus.cc:504] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.695142 5196 leader_election.cc:288] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 election: Requested vote from peers 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183), 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613)
I20260806 18:44:45.695672 4678 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144e0f9bf6b747e9a2bcf79738f00008" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0"
I20260806 18:44:45.695817 4678 raft_consensus.cc:3037] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.695768 4873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "144e0f9bf6b747e9a2bcf79738f00008" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b64d8999cb4689afb25fa8a4897162"
I20260806 18:44:45.695914 4873 raft_consensus.cc:3037] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:45.696995 4873 raft_consensus.cc:2454] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 1.
I20260806 18:44:45.697014 4678 raft_consensus.cc:2454] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 1.
I20260806 18:44:45.697345 4977 leader_election.cc:302] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56b64d8999cb4689afb25fa8a4897162, f0d928d8198046f68f10864efd913d7f; no voters:
I20260806 18:44:45.697549 5196 raft_consensus.cc:2781] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:45.697640 5196 raft_consensus.cc:686] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 1 LEADER]: Becoming Leader. State: Replica: f0d928d8198046f68f10864efd913d7f, State: Running, Role: LEADER
I20260806 18:44:45.697772 5196 consensus_queue.cc:237] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:45.699663 4451 catalog_manager.cc:5495] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f reported cstate change: term changed from 0 to 1, leader changed from <none> to f0d928d8198046f68f10864efd913d7f (127.1.245.67). New cstate: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } health_report { overall_health: HEALTHY } } }
I20260806 18:44:45.716188 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.713078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35970 (local address 127.1.245.67:39961)
0806 18:44:45.713191 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.713194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.713207 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:44:45.713395 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.713398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.713450 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.713524 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:45.713846 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.714497 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.715340 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.715559 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.715650 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.715843 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.715910 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.715958 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.716029 (+ 71us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.716073 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:44:45.725023 5167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.722205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:53565 (local address 127.1.245.67:39961)
0806 18:44:45.722377 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:45.722381 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:45.722395 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:44:45.722454 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:45.722457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:45.722497 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:45.722552 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:45.722756 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.723226 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.724028 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.724241 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.724355 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.724626 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.724708 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.724780 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:45.724865 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 18:44:45.724905 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 18:44:45.725034 5226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:45.722072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:39961 (local address 127.1.245.67:53565)
0806 18:44:45.722244 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:45.722260 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:45.722375 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:45.722578 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:45.722580 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:45.722586 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:44:45.722664 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.722670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.723333 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.723336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.723912 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:45.723920 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:45.724386 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:45.724390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:45.724459 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:45.724492 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:44:45.724802 (+ 310us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:44:45.724808 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:44:45.724856 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 18:44:45.724922 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 18:44:45.730748 4873 raft_consensus.cc:1261] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Refusing update from remote peer f0d928d8198046f68f10864efd913d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:45.731103 4678 raft_consensus.cc:1261] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Refusing update from remote peer f0d928d8198046f68f10864efd913d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:45.731161 5212 consensus_queue.cc:1035] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:45.731470 5212 consensus_queue.cc:1035] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:44:45.744071 5203 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:44:47.299681 4451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:47.299860 4451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:47.302536 4451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: Sending DeleteTablet for 3 replicas of tablet 144e0f9bf6b747e9a2bcf79738f00008
I20260806 18:44:47.303156 4843 tablet_service.cc:1514] Processing DeleteTablet for tablet 144e0f9bf6b747e9a2bcf79738f00008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:46254
I20260806 18:44:47.303130 4651 tablet_service.cc:1514] Processing DeleteTablet for tablet 144e0f9bf6b747e9a2bcf79738f00008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:58700
I20260806 18:44:47.303483 5038 tablet_service.cc:1514] Processing DeleteTablet for tablet 144e0f9bf6b747e9a2bcf79738f00008 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35954
I20260806 18:44:47.303856 5291 tablet_replica.cc:330] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: stopping tablet replica
I20260806 18:44:47.303979 5291 raft_consensus.cc:2229] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:44:47.304258 5291 raft_consensus.cc:2258] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:47.306279 5289 tablet_replica.cc:330] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: stopping tablet replica
I20260806 18:44:47.306445 5289 raft_consensus.cc:2229] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:47.306587 5289 raft_consensus.cc:2258] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:47.306644 5290 tablet_replica.cc:330] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: stopping tablet replica
I20260806 18:44:47.306636 5291 ts_tablet_manager.cc:1905] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:47.306774 5290 raft_consensus.cc:2229] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:47.306928 5290 raft_consensus.cc:2258] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:47.307539 5290 ts_tablet_manager.cc:1905] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:47.309712 5289 ts_tablet_manager.cc:1905] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:47.309887 5291 ts_tablet_manager.cc:1918] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:44:47.309963 5291 log.cc:1196] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/144e0f9bf6b747e9a2bcf79738f00008
I20260806 18:44:47.310207 5290 ts_tablet_manager.cc:1918] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:44:47.310249 5291 ts_tablet_manager.cc:1939] T 144e0f9bf6b747e9a2bcf79738f00008 P f0d928d8198046f68f10864efd913d7f: Deleting consensus metadata
I20260806 18:44:47.310290 5290 log.cc:1196] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/144e0f9bf6b747e9a2bcf79738f00008
I20260806 18:44:47.310542 5290 ts_tablet_manager.cc:1939] T 144e0f9bf6b747e9a2bcf79738f00008 P 56b64d8999cb4689afb25fa8a4897162: Deleting consensus metadata
I20260806 18:44:47.311092 4427 catalog_manager.cc:4841] TS f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961): tablet 144e0f9bf6b747e9a2bcf79738f00008 (table test-table [id=e6aee0d8655545a2a64407037819a04e]) successfully deleted
I20260806 18:44:47.311208 4431 catalog_manager.cc:4841] TS 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613): tablet 144e0f9bf6b747e9a2bcf79738f00008 (table test-table [id=e6aee0d8655545a2a64407037819a04e]) successfully deleted
I20260806 18:44:47.312475 5289 ts_tablet_manager.cc:1918] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:44:47.312549 5289 log.cc:1196] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/144e0f9bf6b747e9a2bcf79738f00008
I20260806 18:44:47.312790 5289 ts_tablet_manager.cc:1939] T 144e0f9bf6b747e9a2bcf79738f00008 P 52be427786d247159029b3cdf492b0c0: Deleting consensus metadata
I20260806 18:44:47.313651 4431 catalog_manager.cc:4841] TS 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183): tablet 144e0f9bf6b747e9a2bcf79738f00008 (table test-table [id=e6aee0d8655545a2a64407037819a04e]) successfully deleted
Aug 06 18:44:47 dist-test-slave-fgb3 krb5kdc[4386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041887, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:44:47 dist-test-slave-fgb3 krb5kdc[4386](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041887, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:47.335850 5301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:47.326995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50406 (local address 127.1.245.126:45825)
0806 18:44:47.327284 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:47.327291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:47.327308 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:44:47.327425 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:47.327428 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:47.327474 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:47.327547 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:47.328225 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:47.328813 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:47.329718 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:47.330017 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:47.330104 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:47.332795 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:47.332798 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:47.332812 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:47.332841 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:47.334201 (+ 1360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:47.334681 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:47.334685 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:47.334687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:47.334787 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:47.335033 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:47.335035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:47.335036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:47.335229 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:47.335343 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:47.335535 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 18:44:47.335654 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":109,"threads_started":1}
I20260806 18:44:47.338717 4451 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:44:47.339187 4451 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:47.344158 4843 tablet_service.cc:1467] Processing CreateTablet for tablet b6610ca7d97845ddb2b1058f05514cc9 (DEFAULT_TABLE table=test-table [id=9dc7acc2826a4b1f99f0567f99b24d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:47.344192 4651 tablet_service.cc:1467] Processing CreateTablet for tablet b6610ca7d97845ddb2b1058f05514cc9 (DEFAULT_TABLE table=test-table [id=9dc7acc2826a4b1f99f0567f99b24d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:47.344501 4651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6610ca7d97845ddb2b1058f05514cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:47.344501 4843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6610ca7d97845ddb2b1058f05514cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:47.346654 5038 tablet_service.cc:1467] Processing CreateTablet for tablet b6610ca7d97845ddb2b1058f05514cc9 (DEFAULT_TABLE table=test-table [id=9dc7acc2826a4b1f99f0567f99b24d01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:47.346962 5038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6610ca7d97845ddb2b1058f05514cc9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:47.347200 5303 tablet_bootstrap.cc:492] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: Bootstrap starting.
I20260806 18:44:47.347577 5307 tablet_bootstrap.cc:492] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: Bootstrap starting.
I20260806 18:44:47.348148 5303 tablet_bootstrap.cc:654] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:47.348414 5307 tablet_bootstrap.cc:654] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:47.349409 5303 tablet_bootstrap.cc:492] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: No bootstrap required, opened a new log
I20260806 18:44:47.349515 5303 ts_tablet_manager.cc:1397] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:47.349907 5311 tablet_bootstrap.cc:492] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: Bootstrap starting.
I20260806 18:44:47.350072 5307 tablet_bootstrap.cc:492] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: No bootstrap required, opened a new log
I20260806 18:44:47.350030 5303 raft_consensus.cc:348] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.350167 5303 raft_consensus.cc:374] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:47.350205 5303 raft_consensus.cc:729] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52be427786d247159029b3cdf492b0c0, State: Initialized, Role: FOLLOWER
I20260806 18:44:47.350359 5307 ts_tablet_manager.cc:1397] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:44:47.350387 5303 consensus_queue.cc:260] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.350826 5311 tablet_bootstrap.cc:654] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:47.350827 5303 ts_tablet_manager.cc:1428] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:44:47.351002 5307 raft_consensus.cc:348] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.351117 5307 raft_consensus.cc:374] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:47.351151 5307 raft_consensus.cc:729] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b64d8999cb4689afb25fa8a4897162, State: Initialized, Role: FOLLOWER
I20260806 18:44:47.351286 5307 consensus_queue.cc:260] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.351692 5307 ts_tablet_manager.cc:1428] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:47.351872 5311 tablet_bootstrap.cc:492] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: No bootstrap required, opened a new log
I20260806 18:44:47.351949 5311 ts_tablet_manager.cc:1397] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 18:44:47.352331 5311 raft_consensus.cc:348] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.352442 5311 raft_consensus.cc:374] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:47.352488 5311 raft_consensus.cc:729] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d928d8198046f68f10864efd913d7f, State: Initialized, Role: FOLLOWER
I20260806 18:44:47.352617 5311 consensus_queue.cc:260] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.352885 5311 ts_tablet_manager.cc:1428] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:44:47.693780 5282 raft_consensus.cc:482] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:47.693954 5282 raft_consensus.cc:504] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.694481 5282 leader_election.cc:288] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183), 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613)
I20260806 18:44:47.695127 4873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6610ca7d97845ddb2b1058f05514cc9" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b64d8999cb4689afb25fa8a4897162" is_pre_election: true
I20260806 18:44:47.695139 4677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6610ca7d97845ddb2b1058f05514cc9" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0" is_pre_election: true
I20260806 18:44:47.695322 4873 raft_consensus.cc:2454] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 0.
I20260806 18:44:47.695324 4677 raft_consensus.cc:2454] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 0.
I20260806 18:44:47.695693 4977 leader_election.cc:302] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52be427786d247159029b3cdf492b0c0, f0d928d8198046f68f10864efd913d7f; no voters:
I20260806 18:44:47.695922 5282 raft_consensus.cc:2781] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:47.695984 5282 raft_consensus.cc:482] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:47.696023 5282 raft_consensus.cc:3037] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:47.697263 5282 raft_consensus.cc:504] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.697576 5282 leader_election.cc:288] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 election: Requested vote from peers 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183), 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613)
I20260806 18:44:47.698009 4677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6610ca7d97845ddb2b1058f05514cc9" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0"
I20260806 18:44:47.698033 4873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6610ca7d97845ddb2b1058f05514cc9" candidate_uuid: "f0d928d8198046f68f10864efd913d7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56b64d8999cb4689afb25fa8a4897162"
I20260806 18:44:47.698129 4677 raft_consensus.cc:3037] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:47.698169 4873 raft_consensus.cc:3037] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:47.699234 4873 raft_consensus.cc:2454] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 1.
I20260806 18:44:47.699256 4677 raft_consensus.cc:2454] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f0d928d8198046f68f10864efd913d7f in term 1.
I20260806 18:44:47.699554 4977 leader_election.cc:302] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52be427786d247159029b3cdf492b0c0, f0d928d8198046f68f10864efd913d7f; no voters:
I20260806 18:44:47.699796 5282 raft_consensus.cc:2781] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:47.699913 5282 raft_consensus.cc:686] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 1 LEADER]: Becoming Leader. State: Replica: f0d928d8198046f68f10864efd913d7f, State: Running, Role: LEADER
I20260806 18:44:47.700085 5282 consensus_queue.cc:237] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.701925 4447 catalog_manager.cc:5495] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f reported cstate change: term changed from 0 to 1, leader changed from <none> to f0d928d8198046f68f10864efd913d7f (127.1.245.67). New cstate: current_term: 1 leader_uuid: "f0d928d8198046f68f10864efd913d7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } health_report { overall_health: HEALTHY } } }
I20260806 18:44:47.726815 5320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:47.723211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35984 (local address 127.1.245.67:39961)
0806 18:44:47.723505 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:47.723513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:47.723525 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:44:47.723564 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:47.723566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:47.723609 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:47.723689 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:47.723991 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:47.724570 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:47.725613 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:47.725856 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:47.725958 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:47.726269 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:47.726411 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:47.726485 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:47.726592 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 18:44:47.726646 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":115,"threads_started":1}
I20260806 18:44:47.732982 4677 raft_consensus.cc:1261] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Refusing update from remote peer f0d928d8198046f68f10864efd913d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:47.732982 4873 raft_consensus.cc:1261] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Refusing update from remote peer f0d928d8198046f68f10864efd913d7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:47.733368 5282 consensus_queue.cc:1035] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:47.733503 5264 consensus_queue.cc:1035] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:47.764992 4447 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:47.765148 4447 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:47.767107 4447 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: Sending DeleteTablet for 3 replicas of tablet b6610ca7d97845ddb2b1058f05514cc9
I20260806 18:44:47.767733 4651 tablet_service.cc:1514] Processing DeleteTablet for tablet b6610ca7d97845ddb2b1058f05514cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:58700
I20260806 18:44:47.767940 5289 tablet_replica.cc:330] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: stopping tablet replica
I20260806 18:44:47.767938 4843 tablet_service.cc:1514] Processing DeleteTablet for tablet b6610ca7d97845ddb2b1058f05514cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:46254
I20260806 18:44:47.768070 5289 raft_consensus.cc:2229] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:47.768105 5290 tablet_replica.cc:330] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: stopping tablet replica
I20260806 18:44:47.768193 5289 raft_consensus.cc:2258] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:47.768199 5290 raft_consensus.cc:2229] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:47.768322 5290 raft_consensus.cc:2258] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:47.768401 4447 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:44:47.768674 5039 tablet_service.cc:1514] Processing DeleteTablet for tablet b6610ca7d97845ddb2b1058f05514cc9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35954
W20260806 18:44:47.768815 4447 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:44:47.768877 5291 tablet_replica.cc:330] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: stopping tablet replica
I20260806 18:44:47.768992 5291 raft_consensus.cc:2229] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:44:47.769176 5291 raft_consensus.cc:2258] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:47.769371 5290 ts_tablet_manager.cc:1905] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:47.769490 5289 ts_tablet_manager.cc:1905] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:47.770594 5291 ts_tablet_manager.cc:1905] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:47.771956 5289 ts_tablet_manager.cc:1918] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:47.772035 5289 log.cc:1196] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/b6610ca7d97845ddb2b1058f05514cc9
I20260806 18:44:47.772022 5290 ts_tablet_manager.cc:1918] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:47.772091 5290 log.cc:1196] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/b6610ca7d97845ddb2b1058f05514cc9
I20260806 18:44:47.772284 5289 ts_tablet_manager.cc:1939] T b6610ca7d97845ddb2b1058f05514cc9 P 52be427786d247159029b3cdf492b0c0: Deleting consensus metadata
I20260806 18:44:47.772337 5290 ts_tablet_manager.cc:1939] T b6610ca7d97845ddb2b1058f05514cc9 P 56b64d8999cb4689afb25fa8a4897162: Deleting consensus metadata
I20260806 18:44:47.773031 5291 ts_tablet_manager.cc:1918] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:44:47.773110 5291 log.cc:1196] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/b6610ca7d97845ddb2b1058f05514cc9
I20260806 18:44:47.773185 4431 catalog_manager.cc:4841] TS 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613): tablet b6610ca7d97845ddb2b1058f05514cc9 (table test-table [id=9dc7acc2826a4b1f99f0567f99b24d01]) successfully deleted
I20260806 18:44:47.773351 5291 ts_tablet_manager.cc:1939] T b6610ca7d97845ddb2b1058f05514cc9 P f0d928d8198046f68f10864efd913d7f: Deleting consensus metadata
I20260806 18:44:47.773337 4431 catalog_manager.cc:4841] TS 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183): tablet b6610ca7d97845ddb2b1058f05514cc9 (table test-table [id=9dc7acc2826a4b1f99f0567f99b24d01]) successfully deleted
I20260806 18:44:47.773957 4427 catalog_manager.cc:4841] TS f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961): tablet b6610ca7d97845ddb2b1058f05514cc9 (table test-table [id=9dc7acc2826a4b1f99f0567f99b24d01]) successfully deleted
I20260806 18:44:47.774811 4651 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4b6c3f2582485788596d99854ca337 (DEFAULT_TABLE table=test-table [id=5155e4fd81e347429bfadc042c73cd39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:47.774843 5039 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4b6c3f2582485788596d99854ca337 (DEFAULT_TABLE table=test-table [id=5155e4fd81e347429bfadc042c73cd39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:47.774904 4843 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4b6c3f2582485788596d99854ca337 (DEFAULT_TABLE table=test-table [id=5155e4fd81e347429bfadc042c73cd39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:44:47.775063 5039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4b6c3f2582485788596d99854ca337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:47.775063 4651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4b6c3f2582485788596d99854ca337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:47.775157 4843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4b6c3f2582485788596d99854ca337. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:47.777201 5307 tablet_bootstrap.cc:492] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: Bootstrap starting.
I20260806 18:44:47.777201 5303 tablet_bootstrap.cc:492] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0: Bootstrap starting.
I20260806 18:44:47.778090 5307 tablet_bootstrap.cc:654] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:47.778090 5303 tablet_bootstrap.cc:654] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:47.778678 5311 tablet_bootstrap.cc:492] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: Bootstrap starting.
I20260806 18:44:47.779168 5307 tablet_bootstrap.cc:492] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: No bootstrap required, opened a new log
I20260806 18:44:47.779263 5307 ts_tablet_manager.cc:1397] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:47.779400 5311 tablet_bootstrap.cc:654] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:47.779654 5307 raft_consensus.cc:348] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.779749 5307 raft_consensus.cc:374] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:47.779783 5307 raft_consensus.cc:729] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56b64d8999cb4689afb25fa8a4897162, State: Initialized, Role: FOLLOWER
I20260806 18:44:47.779904 5307 consensus_queue.cc:260] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.780164 5307 ts_tablet_manager.cc:1428] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:47.780640 5303 tablet_bootstrap.cc:492] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0: No bootstrap required, opened a new log
I20260806 18:44:47.780731 5303 ts_tablet_manager.cc:1397] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 18:44:47.780972 5311 tablet_bootstrap.cc:492] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: No bootstrap required, opened a new log
I20260806 18:44:47.781057 5311 ts_tablet_manager.cc:1397] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 18:44:47.781172 5303 raft_consensus.cc:348] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.781288 5303 raft_consensus.cc:374] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:47.781329 5303 raft_consensus.cc:729] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52be427786d247159029b3cdf492b0c0, State: Initialized, Role: FOLLOWER
I20260806 18:44:47.781419 5311 raft_consensus.cc:348] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.781446 5303 consensus_queue.cc:260] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.781522 5311 raft_consensus.cc:374] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:47.781553 5311 raft_consensus.cc:729] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d928d8198046f68f10864efd913d7f, State: Initialized, Role: FOLLOWER
I20260806 18:44:47.781690 5303 ts_tablet_manager.cc:1428] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:44:47.781669 5311 consensus_queue.cc:260] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.781944 5311 ts_tablet_manager.cc:1428] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:44:47.857600 5314 raft_consensus.cc:482] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:44:47.857748 5314 raft_consensus.cc:504] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.858234 5314 leader_election.cc:288] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183), f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961)
I20260806 18:44:47.858659 5076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4b6c3f2582485788596d99854ca337" candidate_uuid: "56b64d8999cb4689afb25fa8a4897162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d928d8198046f68f10864efd913d7f" is_pre_election: true
I20260806 18:44:47.858692 4677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4b6c3f2582485788596d99854ca337" candidate_uuid: "56b64d8999cb4689afb25fa8a4897162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0" is_pre_election: true
I20260806 18:44:47.858803 5076 raft_consensus.cc:2454] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b64d8999cb4689afb25fa8a4897162 in term 0.
I20260806 18:44:47.858852 4677 raft_consensus.cc:2454] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 56b64d8999cb4689afb25fa8a4897162 in term 0.
I20260806 18:44:47.859112 4776 leader_election.cc:302] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 56b64d8999cb4689afb25fa8a4897162, f0d928d8198046f68f10864efd913d7f; no voters:
I20260806 18:44:47.859336 5314 raft_consensus.cc:2781] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:44:47.859395 5314 raft_consensus.cc:482] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:44:47.859431 5314 raft_consensus.cc:3037] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:47.860402 5314 raft_consensus.cc:504] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.860718 5314 leader_election.cc:288] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [CANDIDATE]: Term 1 election: Requested vote from peers 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183), f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961)
I20260806 18:44:47.861042 4677 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4b6c3f2582485788596d99854ca337" candidate_uuid: "56b64d8999cb4689afb25fa8a4897162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "52be427786d247159029b3cdf492b0c0"
I20260806 18:44:47.861055 5076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4b6c3f2582485788596d99854ca337" candidate_uuid: "56b64d8999cb4689afb25fa8a4897162" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d928d8198046f68f10864efd913d7f"
I20260806 18:44:47.861168 4677 raft_consensus.cc:3037] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:47.861168 5076 raft_consensus.cc:3037] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:47.862296 5076 raft_consensus.cc:2454] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b64d8999cb4689afb25fa8a4897162 in term 1.
I20260806 18:44:47.862293 4677 raft_consensus.cc:2454] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 56b64d8999cb4689afb25fa8a4897162 in term 1.
I20260806 18:44:47.862582 4778 leader_election.cc:302] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52be427786d247159029b3cdf492b0c0, 56b64d8999cb4689afb25fa8a4897162; no voters:
I20260806 18:44:47.862743 5314 raft_consensus.cc:2781] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:47.862932 5314 raft_consensus.cc:686] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 1 LEADER]: Becoming Leader. State: Replica: 56b64d8999cb4689afb25fa8a4897162, State: Running, Role: LEADER
I20260806 18:44:47.863072 5314 consensus_queue.cc:237] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } }
I20260806 18:44:47.864770 4451 catalog_manager.cc:5495] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 reported cstate change: term changed from 0 to 1, leader changed from <none> to 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "56b64d8999cb4689afb25fa8a4897162" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "56b64d8999cb4689afb25fa8a4897162" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 } health_report { overall_health: UNKNOWN } } }
I20260806 18:44:47.888401 5332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:47.885180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46268 (local address 127.1.245.66:45613)
0806 18:44:47.885494 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:47.885503 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:47.885513 (+ 10us) server_negotiation.cc:407] Connection header received
0806 18:44:47.885550 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:47.885553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:47.885592 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:47.885664 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:44:47.885940 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:47.886511 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:47.887342 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:47.887577 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:47.887668 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:47.887933 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:44:47.888009 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:44:47.888067 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:47.888194 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 18:44:47.888249 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":122,"threads_started":1}
I20260806 18:44:47.894302 5076 raft_consensus.cc:1261] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Refusing update from remote peer 56b64d8999cb4689afb25fa8a4897162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:47.894817 5330 consensus_queue.cc:1035] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0d928d8198046f68f10864efd913d7f" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:44:47.894893 4677 raft_consensus.cc:1261] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Refusing update from remote peer 56b64d8999cb4689afb25fa8a4897162: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:44:47.895232 5330 consensus_queue.cc:1035] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [LEADER]: Connected to new peer: Peer: permanent_uuid: "52be427786d247159029b3cdf492b0c0" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:44:47.909006 5259 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 18:44:49.459216 4451 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:49.459399 4451 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50406:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:44:49.461819 4451 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 509abd8a25424c798ca710e94e389b36: Sending DeleteTablet for 3 replicas of tablet 2b4b6c3f2582485788596d99854ca337
I20260806 18:44:49.462409 4651 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b4b6c3f2582485788596d99854ca337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:49 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:58700
I20260806 18:44:49.462517 4843 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b4b6c3f2582485788596d99854ca337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:49 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:46254
I20260806 18:44:49.462749 5365 tablet_replica.cc:330] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0: stopping tablet replica
I20260806 18:44:49.462829 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 4556
I20260806 18:44:49.462832 5039 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b4b6c3f2582485788596d99854ca337 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:44:49 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35954
I20260806 18:44:49.462885 5365 raft_consensus.cc:2229] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:49.462985 5365 raft_consensus.cc:2258] T 2b4b6c3f2582485788596d99854ca337 P 52be427786d247159029b3cdf492b0c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:49.467509 5366 tablet_replica.cc:330] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: stopping tablet replica
I20260806 18:44:49.467680 5366 raft_consensus.cc:2229] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:44:49.467897 5366 raft_consensus.cc:2258] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:49.468520 5366 ts_tablet_manager.cc:1905] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:49.470945 5366 ts_tablet_manager.cc:1918] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:44:49.471019 5366 log.cc:1196] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/2b4b6c3f2582485788596d99854ca337
I20260806 18:44:49.471253 5366 ts_tablet_manager.cc:1939] T 2b4b6c3f2582485788596d99854ca337 P 56b64d8999cb4689afb25fa8a4897162: Deleting consensus metadata
I20260806 18:44:49.472153 4431 catalog_manager.cc:4841] TS 56b64d8999cb4689afb25fa8a4897162 (127.1.245.66:45613): tablet 2b4b6c3f2582485788596d99854ca337 (table test-table [id=5155e4fd81e347429bfadc042c73cd39]) successfully deleted
I20260806 18:44:49.472630 5367 tablet_replica.cc:330] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: stopping tablet replica
I20260806 18:44:49.472754 5367 raft_consensus.cc:2229] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:44:49.472883 5367 raft_consensus.cc:2258] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:44:49.473505 5367 ts_tablet_manager.cc:1905] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:44:49.475581 5367 ts_tablet_manager.cc:1918] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 18:44:49.475646 5367 log.cc:1196] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/2b4b6c3f2582485788596d99854ca337
I20260806 18:44:49.475853 5367 ts_tablet_manager.cc:1939] T 2b4b6c3f2582485788596d99854ca337 P f0d928d8198046f68f10864efd913d7f: Deleting consensus metadata
I20260806 18:44:49.476689 4427 catalog_manager.cc:4841] TS f0d928d8198046f68f10864efd913d7f (127.1.245.67:39961): tablet 2b4b6c3f2582485788596d99854ca337 (table test-table [id=5155e4fd81e347429bfadc042c73cd39]) successfully deleted
W20260806 18:44:49.478406 4431 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:34183 (error 108)
I20260806 18:44:49.478452 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 4758
I20260806 18:44:49.487561 5369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:49.486896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34183 (local address 127.0.0.1:58704)
0806 18:44:49.487306 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:49.487425 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:34183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":94,"threads_started":1}
W20260806 18:44:49.487713 4431 catalog_manager.cc:4584] TS 52be427786d247159029b3cdf492b0c0 (127.1.245.65:34183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b4b6c3f2582485788596d99854ca337: Network error: Client connection negotiation failed: client connection to 127.1.245.65:34183: connect: Connection refused (error 111)
I20260806 18:44:49.495147 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 4954
I20260806 18:44:49.511848 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 4402
2026-08-06T18:44:49Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6860 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:49 dist-test-slave-fgb3 krb5kdc[5373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:49 dist-test-slave-fgb3 krb5kdc[5373](info): set up 2 sockets
Aug 06 18:44:49 dist-test-slave-fgb3 krb5kdc[5373](info): commencing operation
krb5kdc: starting...
W20260806 18:44:50.643631 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:50 dist-test-slave-fgb3 krb5kdc[5373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:50.774482 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:40489
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:40489
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:50.887957 5396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:50.888242 5396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:50.888295 5396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:50.892318 5396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:50.892383 5396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:50.892408 5396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:50.892429 5396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:50.892447 5396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:50.897833 5396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:40489
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:40489
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:50.899037 5396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:50.900098 5396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:50.908933 5401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:50.908942 5404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:50.908985 5396 server_base.cc:1034] running on GCE node
W20260806 18:44:50.908927 5402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:50.909772 5396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:50.910952 5396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:50.912117 5396 hybrid_clock.cc:648] HybridClock initialized: now 1786041890912081 us; error 49 us; skew 500 ppm
Aug 06 18:44:50 dist-test-slave-fgb3 krb5kdc[5373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041890, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:50.918650 5396 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:50.919625 5396 webserver.cc:460] Webserver started at http://127.1.245.126:46373/ using document root <none> and password file <none>
I20260806 18:44:50.920185 5396 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:50.920255 5396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:50.920470 5396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:50.922595 5396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "c71fc220129a430e8250bdf29d9fc945"
format_stamp: "Formatted at 2026-08-06 18:44:50 on dist-test-slave-fgb3"
server_key: "de27e31469e45db02bc385865047a3c8"
server_key_iv: "fc7a5a3570279fe39d0a6c1ad181d3f9"
server_key_version: "encryptionkey@0"
I20260806 18:44:50.923213 5396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "c71fc220129a430e8250bdf29d9fc945"
format_stamp: "Formatted at 2026-08-06 18:44:50 on dist-test-slave-fgb3"
server_key: "de27e31469e45db02bc385865047a3c8"
server_key_iv: "fc7a5a3570279fe39d0a6c1ad181d3f9"
server_key_version: "encryptionkey@0"
I20260806 18:44:50.927160 5396 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:50.929636 5418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:50.930954 5396 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:50.931072 5396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "c71fc220129a430e8250bdf29d9fc945"
format_stamp: "Formatted at 2026-08-06 18:44:50 on dist-test-slave-fgb3"
server_key: "de27e31469e45db02bc385865047a3c8"
server_key_iv: "fc7a5a3570279fe39d0a6c1ad181d3f9"
server_key_version: "encryptionkey@0"
I20260806 18:44:50.931165 5396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:50.957041 5396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:50.960333 5396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:50.960551 5396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:50.968305 5396 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:40489
I20260806 18:44:50.968333 5497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:40489 every 8 connection(s)
I20260806 18:44:50.969313 5396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:50.971709 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 5396
I20260806 18:44:50.971817 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:50.972112 2005 external_mini_cluster.cc:1428] Setting key f40dc93e43ce779a01e9afac7a6d89e2
I20260806 18:44:50.972486 5498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:50.979295 5498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945: Bootstrap starting.
Aug 06 18:44:50 dist-test-slave-fgb3 krb5kdc[5373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:50.981976 5498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:50.982764 5498 log.cc:824] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:50.984877 5498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945: No bootstrap required, opened a new log
I20260806 18:44:50.986940 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:50.973523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56188 (local address 127.1.245.126:40489)
0806 18:44:50.973966 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:50.973983 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:50.974022 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:44:50.974736 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:50.974755 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:50.975075 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:50.975354 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:50.976305 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:50.977326 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:50.978549 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:50.979034 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:50.979170 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:50.981971 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:50.981977 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:50.981991 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:50.982024 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:50.983805 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:50.984304 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:50.984310 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:50.984314 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:50.984439 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:50.984759 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:50.984762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:50.984763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:50.985203 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:50.985383 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:50.985725 (+ 342us) server_negotiation.cc:299] Negotiation successful
0806 18:44:50.986130 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":138,"threads_started":1}
I20260806 18:44:50.988796 5498 raft_consensus.cc:348] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71fc220129a430e8250bdf29d9fc945" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40489 } }
I20260806 18:44:50.989064 5498 raft_consensus.cc:374] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:50.989126 5498 raft_consensus.cc:729] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c71fc220129a430e8250bdf29d9fc945, State: Initialized, Role: FOLLOWER
I20260806 18:44:50.989528 5498 consensus_queue.cc:260] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71fc220129a430e8250bdf29d9fc945" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40489 } }
I20260806 18:44:50.989655 5498 raft_consensus.cc:388] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:50.989718 5498 raft_consensus.cc:482] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:50.989822 5498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:50.990778 5498 raft_consensus.cc:504] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71fc220129a430e8250bdf29d9fc945" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40489 } }
I20260806 18:44:50.991093 5498 leader_election.cc:302] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c71fc220129a430e8250bdf29d9fc945; no voters:
I20260806 18:44:50.991432 5498 leader_election.cc:288] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:50.991572 5504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:50.991833 5504 raft_consensus.cc:686] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [term 1 LEADER]: Becoming Leader. State: Replica: c71fc220129a430e8250bdf29d9fc945, State: Running, Role: LEADER
I20260806 18:44:50.992138 5504 consensus_queue.cc:237] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71fc220129a430e8250bdf29d9fc945" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40489 } }
I20260806 18:44:50.992623 5498 sys_catalog.cc:564] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:50.993824 5505 sys_catalog.cc:455] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c71fc220129a430e8250bdf29d9fc945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71fc220129a430e8250bdf29d9fc945" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40489 } } }
I20260806 18:44:50.993788 5507 sys_catalog.cc:455] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c71fc220129a430e8250bdf29d9fc945. Latest consensus state: current_term: 1 leader_uuid: "c71fc220129a430e8250bdf29d9fc945" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c71fc220129a430e8250bdf29d9fc945" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40489 } } }
I20260806 18:44:50.993919 5505 sys_catalog.cc:458] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:50.993920 5507 sys_catalog.cc:458] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945 [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:50.994366 5516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:50.997200 5516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:51.002547 5516 catalog_manager.cc:1347] Generated new cluster ID: a4ae20d5757e4c9093bbe0f94cc96982
I20260806 18:44:51.002622 5516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:51.014796 5516 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:51.015822 5516 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:51.025045 5516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c71fc220129a430e8250bdf29d9fc945: Generated new TSK 0
I20260806 18:44:51.025857 5516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:44:51.062573 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40489
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:51.176000 5533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:51.176265 5533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:51.176323 5533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:51.180330 5533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:51.180394 5533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:51.180474 5533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:44:51.185792 5533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:51.186915 5533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:51.187990 5533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:51.198166 5540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:51.198246 5549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:51.198161 5543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:51.198207 5533 server_base.cc:1034] running on GCE node
I20260806 18:44:51.198912 5533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:51.199601 5533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:51.200800 5533 hybrid_clock.cc:648] HybridClock initialized: now 1786041891200782 us; error 28 us; skew 500 ppm
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[5373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041891, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:51.203594 5533 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:44:51.204500 5533 webserver.cc:460] Webserver started at http://127.1.245.65:35397/ using document root <none> and password file <none>
I20260806 18:44:51.205058 5533 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:51.205124 5533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:51.205339 5533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:51.207463 5533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "d2a2348f47514b038fcb6043df5b9714"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "d8e0f7d83cdc6f7c39071a45b64aa528"
server_key_iv: "b85fa861f1ceba1e392260c5cfc487c5"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.208036 5533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "d2a2348f47514b038fcb6043df5b9714"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "d8e0f7d83cdc6f7c39071a45b64aa528"
server_key_iv: "b85fa861f1ceba1e392260c5cfc487c5"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.212093 5533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:44:51.214588 5558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.215809 5533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:44:51.215924 5533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "d2a2348f47514b038fcb6043df5b9714"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "d8e0f7d83cdc6f7c39071a45b64aa528"
server_key_iv: "b85fa861f1ceba1e392260c5cfc487c5"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.216022 5533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:51.225572 5533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:51.228878 5533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:51.229090 5533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:51.229748 5533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:51.230669 5533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:51.230732 5533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.230795 5533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:51.230824 5533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.241164 5533 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:38593
I20260806 18:44:51.241183 5745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:38593 every 8 connection(s)
I20260806 18:44:51.242162 5533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:44:51.250123 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 5533
I20260806 18:44:51.250253 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:44:51.250608 2005 external_mini_cluster.cc:1428] Setting key f2caddf216f64556132d306f9c608f02
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[5373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041891, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:51.256650 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:51.244157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:50919 (local address 127.1.245.126:40489)
0806 18:44:51.244295 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:51.244299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:51.245150 (+ 851us) server_negotiation.cc:407] Connection header received
0806 18:44:51.246503 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:51.246507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:51.246562 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:51.246646 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:51.247834 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.248394 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.249575 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.249829 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.249922 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:51.252792 (+ 2870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:51.252795 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:51.252797 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:51.252822 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:51.254300 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:51.254943 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:51.254946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:51.254947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:51.255044 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:51.255438 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:51.255441 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:51.255442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:51.255643 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:51.255743 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:51.256343 (+ 600us) server_negotiation.cc:299] Negotiation successful
0806 18:44:51.256472 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:44:51.257649 5749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:51.244469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40489 (local address 127.1.245.65:50919)
0806 18:44:51.244974 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:51.245021 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:51.246263 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:51.246800 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:51.246806 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:51.247156 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:51.247666 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:51.247678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.248523 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.248525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:51.249438 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:51.249448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.250021 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.250026 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:51.250243 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:51.250924 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:51.250942 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:51.252608 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:51.254441 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:51.254447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:51.254463 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:51.254811 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:51.255152 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:51.255155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:51.255157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:51.255329 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:51.255763 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:51.255769 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:51.256061 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 18:44:51.256277 (+ 216us) client_negotiation.cc:240] Negotiation successful
0806 18:44:51.256614 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":129,"threads_started":1}
I20260806 18:44:51.259092 5746 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40489
I20260806 18:44:51.259352 5746 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:51.259883 5746 heartbeater.cc:511] Master 127.1.245.126:40489 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:44:51.261464 5440 ts_manager.cc:194] Registered new tserver with Master: d2a2348f47514b038fcb6043df5b9714 (127.1.245.65:38593)
I20260806 18:44:51.262706 5440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:50919
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:44:51.281536 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40489
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:51.397805 5753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:51.398104 5753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:51.398149 5753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:51.402230 5753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:51.402304 5753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:51.402406 5753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:44:51.407734 5753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:51.408800 5753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:51.409806 5753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:51.419729 5765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:51.419775 5762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:51.419889 5753 server_base.cc:1034] running on GCE node
W20260806 18:44:51.419751 5767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:51.420511 5753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:51.421136 5753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:51.422338 5753 hybrid_clock.cc:648] HybridClock initialized: now 1786041891422290 us; error 43 us; skew 500 ppm
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[5373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041891, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:51.425038 5753 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:44:51.425913 5753 webserver.cc:460] Webserver started at http://127.1.245.66:36665/ using document root <none> and password file <none>
I20260806 18:44:51.426494 5753 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:51.426568 5753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:51.426788 5753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:51.428836 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "b4eb4f4395554f0f8adfc30fa6f8139e"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "cfd4268367f75df8a76c946c7e64bdd3"
server_key_iv: "c0a1f466ab91762f476a22a322ae1000"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.429404 5753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "b4eb4f4395554f0f8adfc30fa6f8139e"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "cfd4268367f75df8a76c946c7e64bdd3"
server_key_iv: "c0a1f466ab91762f476a22a322ae1000"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.433286 5753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:44:51.436157 5776 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.437604 5753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:44:51.437742 5753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "b4eb4f4395554f0f8adfc30fa6f8139e"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "cfd4268367f75df8a76c946c7e64bdd3"
server_key_iv: "c0a1f466ab91762f476a22a322ae1000"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.437855 5753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:51.456066 5753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:51.459086 5753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:51.459259 5753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:51.459870 5753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:51.460723 5753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:51.460783 5753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.460824 5753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:51.460844 5753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.469748 5753 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:44017
I20260806 18:44:51.469836 5940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:44017 every 8 connection(s)
I20260806 18:44:51.470698 5753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:44:51.478569 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 5753
I20260806 18:44:51.478688 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:44:51.479014 2005 external_mini_cluster.cc:1428] Setting key e5fe0ca94ddd77d28d46be46544e97f9
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[5373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041891, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:51.485118 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:51.472585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:38367 (local address 127.1.245.126:40489)
0806 18:44:51.472716 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:51.472720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:51.473524 (+ 804us) server_negotiation.cc:407] Connection header received
0806 18:44:51.474913 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:51.474917 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:51.474978 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:51.475056 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:51.476303 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.476836 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.478019 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.478274 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.478403 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:51.481150 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:51.481154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:51.481156 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:51.481182 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:51.482576 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:51.483190 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:51.483194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:51.483195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:51.483392 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:51.483854 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:51.483858 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:51.483859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:51.484089 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:51.484200 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:51.484801 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 18:44:51.484943 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:44:51.485991 5945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:51.472899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40489 (local address 127.1.245.66:38367)
0806 18:44:51.473361 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:51.473408 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:51.474681 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:51.475265 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:51.475271 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:51.475648 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:51.476143 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:51.476156 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.476968 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.476971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:51.477866 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:51.477876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.478447 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.478452 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:51.478630 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:51.479299 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:51.479320 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:51.480981 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:51.482697 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:51.482704 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:51.482723 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:51.483078 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:51.483506 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:51.483531 (+ 25us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:51.483533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:51.483719 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:51.484236 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:51.484242 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:51.484516 (+ 274us) client_negotiation.cc:769] Sending connection context
0806 18:44:51.484697 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:44:51.485012 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260806 18:44:51.487422 5941 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40489
I20260806 18:44:51.487686 5941 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:51.488248 5941 heartbeater.cc:511] Master 127.1.245.126:40489 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:44:51.489403 5440 ts_manager.cc:194] Registered new tserver with Master: b4eb4f4395554f0f8adfc30fa6f8139e (127.1.245.66:44017)
I20260806 18:44:51.489995 5440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:38367
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:44:51.510583 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40489
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:51.625725 5949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:51.625994 5949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:51.626057 5949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:51.630107 5949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:51.630190 5949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:51.630280 5949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:44:51.635799 5949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40489
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:51.636931 5949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:51.637964 5949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:51.648149 5955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:51.648257 5958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:51.648339 5949 server_base.cc:1034] running on GCE node
W20260806 18:44:51.648522 5956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:51.648988 5949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:51.649618 5949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:51.650784 5949 hybrid_clock.cc:648] HybridClock initialized: now 1786041891650757 us; error 38 us; skew 500 ppm
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[5373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041891, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:51.653611 5949 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:44:51.654592 5949 webserver.cc:460] Webserver started at http://127.1.245.67:46217/ using document root <none> and password file <none>
I20260806 18:44:51.655155 5949 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:51.655222 5949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:51.655423 5949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:51.657646 5949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "2a3e1ed3355d405aaff8515f2b9cbdf6"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "bb517e1d1eefa678c62a28fefa6929f8"
server_key_iv: "91becbc251193cb07042da0d1175d9d5"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.658350 5949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "2a3e1ed3355d405aaff8515f2b9cbdf6"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "bb517e1d1eefa678c62a28fefa6929f8"
server_key_iv: "91becbc251193cb07042da0d1175d9d5"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.662398 5949 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:44:51.664914 5965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.666239 5949 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:44:51.666424 5949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "2a3e1ed3355d405aaff8515f2b9cbdf6"
format_stamp: "Formatted at 2026-08-06 18:44:51 on dist-test-slave-fgb3"
server_key: "bb517e1d1eefa678c62a28fefa6929f8"
server_key_iv: "91becbc251193cb07042da0d1175d9d5"
server_key_version: "encryptionkey@0"
I20260806 18:44:51.666540 5949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:51.686607 5949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:51.690255 5949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:51.690500 5949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:51.691184 5949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:51.692148 5949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:51.692205 5949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.692266 5949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:51.692301 5949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:51.701960 5949 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:43923
I20260806 18:44:51.701987 6117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:43923 every 8 connection(s)
I20260806 18:44:51.702980 5949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:44:51.708027 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 5949
I20260806 18:44:51.708153 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:44:51.708459 2005 external_mini_cluster.cc:1428] Setting key 917b543734c58c52ec0002d4d04303d2
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[5373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041891, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:51.717127 5501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:51.704977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:53785 (local address 127.1.245.126:40489)
0806 18:44:51.705141 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:51.705145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:51.705977 (+ 832us) server_negotiation.cc:407] Connection header received
0806 18:44:51.707386 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:51.707389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:51.707450 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:51.707522 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:51.709026 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.709718 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.711073 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.711286 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.711376 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:51.713922 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:51.713925 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:51.713946 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:51.713971 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:51.715439 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:51.715898 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:51.715901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:51.715902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:51.715968 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:51.716237 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:51.716238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:51.716240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:51.716399 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:51.716468 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:51.716919 (+ 451us) server_negotiation.cc:299] Negotiation successful
0806 18:44:51.717008 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 18:44:51.717995 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:51.705333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40489 (local address 127.1.245.67:53785)
0806 18:44:51.705811 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:51.705855 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:51.707151 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:51.707730 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:51.707737 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:51.708099 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:51.708831 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:51.708857 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.709874 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.709879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:51.710960 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:51.710972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:51.711411 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:51.711414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:51.711556 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:51.712201 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:51.712223 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:51.713736 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:51.715547 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:51.715552 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:51.715564 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:51.715820 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:51.716039 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:51.716041 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:51.716043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:51.716168 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:51.716490 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:51.716495 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:51.716724 (+ 229us) client_negotiation.cc:769] Sending connection context
0806 18:44:51.716886 (+ 162us) client_negotiation.cc:240] Negotiation successful
0806 18:44:51.717135 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":121,"threads_started":1}
I20260806 18:44:51.719244 6118 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40489
I20260806 18:44:51.719498 6118 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:51.720045 6118 heartbeater.cc:511] Master 127.1.245.126:40489 requested a full tablet report, sending...
I20260806 18:44:51.721120 5440 ts_manager.cc:194] Registered new tserver with Master: 2a3e1ed3355d405aaff8515f2b9cbdf6 (127.1.245.67:43923)
I20260806 18:44:51.721825 5440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:53785
I20260806 18:44:51.722549 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:44:51.737079 5501 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:44:51.735540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56202 (local address 127.1.245.126:40489)
0806 18:44:51.735691 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:51.735696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:51.735959 (+ 263us) server_negotiation.cc:407] Connection header received
0806 18:44:51.736120 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:51.736124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:51.736173 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:51.736240 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:51.736970 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56202: BlockingRecv error: recv got EOF from 127.0.0.1:56202 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260806 18:44:51.738080 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 5533
I20260806 18:44:51.749099 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 5753
I20260806 18:44:51.759696 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 5949
I20260806 18:44:51.774946 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 5396
2026-08-06T18:44:51Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2258 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[6132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[6132](info): set up 2 sockets
Aug 06 18:44:51 dist-test-slave-fgb3 krb5kdc[6132](info): commencing operation
krb5kdc: starting...
W20260806 18:44:52.906670 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:52 dist-test-slave-fgb3 krb5kdc[6132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:53.033843 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33263
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:45863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:33263
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:53.146329 6151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:53.146615 6151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:53.146672 6151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:53.150671 6151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:53.150734 6151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:53.150760 6151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:53.150782 6151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:53.150800 6151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:53.156111 6151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:45863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:33263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:53.157234 6151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:53.158340 6151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:53.166882 6156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:53.166903 6157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:53.167068 6151 server_base.cc:1034] running on GCE node
W20260806 18:44:53.166882 6160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:53.167798 6151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:53.168953 6151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:53.170117 6151 hybrid_clock.cc:648] HybridClock initialized: now 1786041893170081 us; error 48 us; skew 500 ppm
Aug 06 18:44:53 dist-test-slave-fgb3 krb5kdc[6132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041893, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:53.172955 6151 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:53.173898 6151 webserver.cc:460] Webserver started at http://127.1.245.126:36149/ using document root <none> and password file <none>
I20260806 18:44:53.174523 6151 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:53.174588 6151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:53.174784 6151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:53.176877 6151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "b8795e1be9c84b65a3983fa41da20a5f"
format_stamp: "Formatted at 2026-08-06 18:44:53 on dist-test-slave-fgb3"
server_key: "2e5dfb52438ff62fec617dbdb479ea13"
server_key_iv: "eaffec5e1e5f91462cc35c8ecf0e34ad"
server_key_version: "encryptionkey@0"
I20260806 18:44:53.177441 6151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "b8795e1be9c84b65a3983fa41da20a5f"
format_stamp: "Formatted at 2026-08-06 18:44:53 on dist-test-slave-fgb3"
server_key: "2e5dfb52438ff62fec617dbdb479ea13"
server_key_iv: "eaffec5e1e5f91462cc35c8ecf0e34ad"
server_key_version: "encryptionkey@0"
I20260806 18:44:53.181396 6151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 18:44:53.184067 6171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:53.185393 6151 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:44:53.185528 6151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "b8795e1be9c84b65a3983fa41da20a5f"
format_stamp: "Formatted at 2026-08-06 18:44:53 on dist-test-slave-fgb3"
server_key: "2e5dfb52438ff62fec617dbdb479ea13"
server_key_iv: "eaffec5e1e5f91462cc35c8ecf0e34ad"
server_key_version: "encryptionkey@0"
I20260806 18:44:53.185633 6151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:53.201083 6151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:53.204550 6151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:53.204743 6151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:53.211937 6151 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:33263
I20260806 18:44:53.212198 6237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:33263 every 8 connection(s)
I20260806 18:44:53.212996 6151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:53.216043 6238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:53.220875 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 6151
I20260806 18:44:53.220990 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:53.221244 2005 external_mini_cluster.cc:1428] Setting key 0477d17869a5dc05c64b57979e53c039
I20260806 18:44:53.222168 6238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f: Bootstrap starting.
I20260806 18:44:53.225323 6238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:53.226248 6238 log.cc:824] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:53.228546 6238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f: No bootstrap required, opened a new log
Aug 06 18:44:53 dist-test-slave-fgb3 krb5kdc[6132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041892, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:53.234158 6238 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8795e1be9c84b65a3983fa41da20a5f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33263 } }
I20260806 18:44:53.234475 6238 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:53.234566 6238 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8795e1be9c84b65a3983fa41da20a5f, State: Initialized, Role: FOLLOWER
I20260806 18:44:53.235060 6238 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8795e1be9c84b65a3983fa41da20a5f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33263 } }
I20260806 18:44:53.235203 6238 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:53.235273 6238 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:53.235371 6238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:53.236363 6238 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8795e1be9c84b65a3983fa41da20a5f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33263 } }
I20260806 18:44:53.236711 6238 leader_election.cc:302] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8795e1be9c84b65a3983fa41da20a5f; no voters:
I20260806 18:44:53.237053 6238 leader_election.cc:288] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:53.237169 6246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:53.237344 6246 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [term 1 LEADER]: Becoming Leader. State: Replica: b8795e1be9c84b65a3983fa41da20a5f, State: Running, Role: LEADER
I20260806 18:44:53.237368 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:53.222810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37830 (local address 127.1.245.126:33263)
0806 18:44:53.223326 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:53.223352 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:53.223401 (+ 49us) server_negotiation.cc:407] Connection header received
0806 18:44:53.224537 (+ 1136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:53.224565 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:53.225003 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:53.225425 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:53.226224 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:53.227582 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:53.228604 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:53.229286 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:53.229465 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:53.231956 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:53.231962 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:53.231979 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:53.232009 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:53.234027 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:53.234560 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:53.234567 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:53.234573 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:53.234736 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:53.235029 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:53.235032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:53.235034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:53.235550 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:53.235756 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:53.236085 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 18:44:53.236388 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":139,"threads_started":1}
I20260806 18:44:53.237726 6246 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8795e1be9c84b65a3983fa41da20a5f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33263 } }
I20260806 18:44:53.238554 6238 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:53.239709 6248 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8795e1be9c84b65a3983fa41da20a5f. Latest consensus state: current_term: 1 leader_uuid: "b8795e1be9c84b65a3983fa41da20a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8795e1be9c84b65a3983fa41da20a5f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33263 } } }
I20260806 18:44:53.239848 6248 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:53.240211 6247 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8795e1be9c84b65a3983fa41da20a5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8795e1be9c84b65a3983fa41da20a5f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33263 } } }
I20260806 18:44:53.240391 6256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:53.240376 6247 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:53.241025 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:44:53.244060 6256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:53.250572 6256 catalog_manager.cc:1347] Generated new cluster ID: fb59558db4684a759c0c9fbacb561e76
I20260806 18:44:53.250711 6256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:53.257880 6256 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:53.259222 6256 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:53.263597 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:53.258164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37834 (local address 127.1.245.126:33263)
0806 18:44:53.258303 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:53.258309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:53.258809 (+ 500us) server_negotiation.cc:407] Connection header received
0806 18:44:53.258946 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:53.258950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:53.259289 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:53.259397 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:53.260055 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:53.260612 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:53.261471 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:53.261713 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:53.261802 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:53.262689 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:53.262692 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:53.262694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:44:53.263136 (+ 442us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:53.263166 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:44:53.263182 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:53.263261 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:53.263386 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 18:44:53.263449 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:44:53.265633 6256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8795e1be9c84b65a3983fa41da20a5f: Generated new TSK 0
I20260806 18:44:53.266332 6256 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:44:53.274765 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 6151
2026-08-06T18:44:53Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1502 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:53 dist-test-slave-fgb3 krb5kdc[6283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:53 dist-test-slave-fgb3 krb5kdc[6283](info): set up 2 sockets
Aug 06 18:44:53 dist-test-slave-fgb3 krb5kdc[6283](info): commencing operation
krb5kdc: starting...
W20260806 18:44:54.414635 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:54 dist-test-slave-fgb3 krb5kdc[6283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:54.541558 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:38757
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:38757
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:54.656433 6299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:54.656703 6299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:54.656745 6299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:54.660912 6299 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:54.660983 6299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:54.661002 6299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:54.661018 6299 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:54.661032 6299 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:54.666445 6299 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:38757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:38757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:54.667511 6299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:54.668525 6299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:54.677126 6307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:54.677145 6304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:54.677315 6299 server_base.cc:1034] running on GCE node
W20260806 18:44:54.677160 6305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:54.678010 6299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:54.679191 6299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:54.680387 6299 hybrid_clock.cc:648] HybridClock initialized: now 1786041894680342 us; error 84 us; skew 500 ppm
Aug 06 18:44:54 dist-test-slave-fgb3 krb5kdc[6283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:54.683226 6299 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:54.684183 6299 webserver.cc:460] Webserver started at http://127.1.245.126:33431/ using document root <none> and password file <none>
I20260806 18:44:54.684741 6299 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:54.684834 6299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:54.685009 6299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:54.687170 6299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "55853bbc09994c9da2111fed26e7868d"
format_stamp: "Formatted at 2026-08-06 18:44:54 on dist-test-slave-fgb3"
server_key: "104029fc8afab7fe3398ce131c601f20"
server_key_iv: "3f423db2cb49b0249aa654c059cd4d1a"
server_key_version: "encryptionkey@0"
I20260806 18:44:54.687691 6299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "55853bbc09994c9da2111fed26e7868d"
format_stamp: "Formatted at 2026-08-06 18:44:54 on dist-test-slave-fgb3"
server_key: "104029fc8afab7fe3398ce131c601f20"
server_key_iv: "3f423db2cb49b0249aa654c059cd4d1a"
server_key_version: "encryptionkey@0"
I20260806 18:44:54.691581 6299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:54.694427 6316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:54.695859 6299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:44:54.695978 6299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "55853bbc09994c9da2111fed26e7868d"
format_stamp: "Formatted at 2026-08-06 18:44:54 on dist-test-slave-fgb3"
server_key: "104029fc8afab7fe3398ce131c601f20"
server_key_iv: "3f423db2cb49b0249aa654c059cd4d1a"
server_key_version: "encryptionkey@0"
I20260806 18:44:54.696075 6299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:54.711481 6299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:54.714820 6299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:54.714999 6299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:54.722227 6299 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:38757
I20260806 18:44:54.722553 6384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:38757 every 8 connection(s)
I20260806 18:44:54.723248 6299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:54.726217 6385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:54.729055 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 6299
I20260806 18:44:54.729184 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:54.729496 2005 external_mini_cluster.cc:1428] Setting key 3a6a03d6a0d09dd419b2e439364a350a
I20260806 18:44:54.732420 6385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d: Bootstrap starting.
I20260806 18:44:54.734623 6385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:54.735339 6385 log.cc:824] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:54.737331 6385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d: No bootstrap required, opened a new log
Aug 06 18:44:54 dist-test-slave-fgb3 krb5kdc[6283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:54.740972 6385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55853bbc09994c9da2111fed26e7868d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38757 } }
I20260806 18:44:54.741247 6385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:54.741307 6385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55853bbc09994c9da2111fed26e7868d, State: Initialized, Role: FOLLOWER
I20260806 18:44:54.741796 6385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55853bbc09994c9da2111fed26e7868d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38757 } }
I20260806 18:44:54.741942 6385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:54.742015 6385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:54.742118 6385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:54.743218 6385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55853bbc09994c9da2111fed26e7868d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38757 } }
I20260806 18:44:54.743563 6385 leader_election.cc:302] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55853bbc09994c9da2111fed26e7868d; no voters:
I20260806 18:44:54.743888 6385 leader_election.cc:288] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:54.743960 6393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:54.744155 6393 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [term 1 LEADER]: Becoming Leader. State: Replica: 55853bbc09994c9da2111fed26e7868d, State: Running, Role: LEADER
I20260806 18:44:54.744519 6393 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55853bbc09994c9da2111fed26e7868d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38757 } }
I20260806 18:44:54.744848 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:54.731020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.1.245.126:38757)
0806 18:44:54.731517 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:54.731538 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:54.731582 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:44:54.732214 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:54.732232 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:54.732635 (+ 403us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:54.732951 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:54.733770 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:54.734873 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:54.735799 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:54.736393 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:54.736547 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:54.738963 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:54.738969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:54.738986 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:54.739017 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:54.741607 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:54.742076 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:54.742082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:54.742089 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:54.742233 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:54.742549 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:54.742553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:54.742555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:54.743068 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:54.743278 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:54.743568 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 18:44:54.743879 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":157,"threads_started":1}
I20260806 18:44:54.745203 6385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:54.747154 6398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55853bbc09994c9da2111fed26e7868d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55853bbc09994c9da2111fed26e7868d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38757 } } }
I20260806 18:44:54.747296 6398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:54.747463 6396 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55853bbc09994c9da2111fed26e7868d. Latest consensus state: current_term: 1 leader_uuid: "55853bbc09994c9da2111fed26e7868d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55853bbc09994c9da2111fed26e7868d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38757 } } }
I20260806 18:44:54.748096 6396 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d [sys.catalog]: This master's current role is: LEADER
W20260806 18:44:54.750064 6413 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:54.750131 6413 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:54.750216 6417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:54.751221 6417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:54.756855 6417 catalog_manager.cc:1347] Generated new cluster ID: bbf02d76c9844fab92ad863a90a1157d
I20260806 18:44:54.756927 6417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:54.779944 6417 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:54.780917 6417 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:54.786044 6417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55853bbc09994c9da2111fed26e7868d: Generated new TSK 0
I20260806 18:44:54.786813 6417 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:44:54.822777 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38757
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:54.937702 6424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:54.938001 6424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:54.938067 6424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:54.942116 6424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:54.942193 6424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:54.942281 6424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:44:54.947870 6424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:54.949111 6424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:54.950204 6424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:54.960202 6431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:54.960263 6424 server_base.cc:1034] running on GCE node
W20260806 18:44:54.960163 6433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:54.960158 6430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:54.960840 6424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:54.961529 6424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:54.962702 6424 hybrid_clock.cc:648] HybridClock initialized: now 1786041894962689 us; error 41 us; skew 500 ppm
Aug 06 18:44:54 dist-test-slave-fgb3 krb5kdc[6283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:54.965566 6424 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:44:54.966616 6424 webserver.cc:460] Webserver started at http://127.1.245.65:35961/ using document root <none> and password file <none>
I20260806 18:44:54.967300 6424 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:54.967368 6424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:54.967540 6424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:54.969609 6424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "b1775b3fc7aa45c7bae8ef953c90c088"
format_stamp: "Formatted at 2026-08-06 18:44:54 on dist-test-slave-fgb3"
server_key: "a80964237aa8226bf91a72a3584d0fad"
server_key_iv: "4ebb58d62459a32f678829a1419b248a"
server_key_version: "encryptionkey@0"
I20260806 18:44:54.970137 6424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "b1775b3fc7aa45c7bae8ef953c90c088"
format_stamp: "Formatted at 2026-08-06 18:44:54 on dist-test-slave-fgb3"
server_key: "a80964237aa8226bf91a72a3584d0fad"
server_key_iv: "4ebb58d62459a32f678829a1419b248a"
server_key_version: "encryptionkey@0"
I20260806 18:44:54.973889 6424 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:44:54.976558 6441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:54.977859 6424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:44:54.978001 6424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "b1775b3fc7aa45c7bae8ef953c90c088"
format_stamp: "Formatted at 2026-08-06 18:44:54 on dist-test-slave-fgb3"
server_key: "a80964237aa8226bf91a72a3584d0fad"
server_key_iv: "4ebb58d62459a32f678829a1419b248a"
server_key_version: "encryptionkey@0"
I20260806 18:44:54.978108 6424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:54.995368 6424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:54.998719 6424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:54.998939 6424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:54.999655 6424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:55.000587 6424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:55.000640 6424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:55.000705 6424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:55.000734 6424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:55.009912 6424 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:37681
I20260806 18:44:55.009984 6621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:37681 every 8 connection(s)
I20260806 18:44:55.010936 6424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:44:55.020321 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 6424
I20260806 18:44:55.020466 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:44:55.020861 2005 external_mini_cluster.cc:1428] Setting key 82234e0950820841d330588972672587
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[6283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041894, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:55.026463 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:55.013073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:50433 (local address 127.1.245.126:38757)
0806 18:44:55.013242 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:55.013247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:55.014116 (+ 869us) server_negotiation.cc:407] Connection header received
0806 18:44:55.015576 (+ 1460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:55.015580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:55.015645 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:55.015721 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:55.017021 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.017800 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.019175 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.019435 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.019547 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:55.022730 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:55.022734 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:55.022736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:55.022762 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:55.024222 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:55.024787 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:55.024790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:55.024791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:55.024873 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:55.025233 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:55.025237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:55.025238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:55.025438 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:55.025524 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:55.026106 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 18:44:55.026243 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:44:55.027326 6625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:55.013447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38757 (local address 127.1.245.65:50433)
0806 18:44:55.013947 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:55.013992 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:55.015354 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:55.015930 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:55.015937 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:55.016324 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:55.016851 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:55.016867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.018000 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.018003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:55.019028 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:55.019041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.019620 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.019625 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:55.019824 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:55.020572 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:55.020599 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:55.022536 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:55.024367 (+ 1831us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:55.024374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:55.024394 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:55.024672 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:55.024965 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:55.024968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:55.024971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:55.025130 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:55.025548 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:55.025556 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:55.025846 (+ 290us) client_negotiation.cc:769] Sending connection context
0806 18:44:55.026025 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 18:44:55.026428 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":122,"threads_started":1}
I20260806 18:44:55.028766 6623 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38757
I20260806 18:44:55.029026 6623 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:55.029551 6623 heartbeater.cc:511] Master 127.1.245.126:38757 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:44:55.031294 6335 ts_manager.cc:194] Registered new tserver with Master: b1775b3fc7aa45c7bae8ef953c90c088 (127.1.245.65:37681)
I20260806 18:44:55.032496 6335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:50433
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:44:55.050591 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38757
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:55.164942 6630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:55.165231 6630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:55.165277 6630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:55.169368 6630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:55.169440 6630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:55.169498 6630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:44:55.174817 6630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:55.175916 6630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:55.176968 6630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:55.186853 6637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:55.186849 6640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:55.186895 6638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:55.186909 6630 server_base.cc:1034] running on GCE node
I20260806 18:44:55.187554 6630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:55.188216 6630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:55.189394 6630 hybrid_clock.cc:648] HybridClock initialized: now 1786041895189374 us; error 31 us; skew 500 ppm
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[6283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:55.192227 6630 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:44:55.193152 6630 webserver.cc:460] Webserver started at http://127.1.245.66:34593/ using document root <none> and password file <none>
I20260806 18:44:55.193775 6630 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:55.193861 6630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:55.194077 6630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:55.196242 6630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "ebb8a2df510c46649c3ab05050c5c8cb"
format_stamp: "Formatted at 2026-08-06 18:44:55 on dist-test-slave-fgb3"
server_key: "83c7959925458888b90225cd27745687"
server_key_iv: "4ae62a93053480fdfa7ceaeb98f0067e"
server_key_version: "encryptionkey@0"
I20260806 18:44:55.196805 6630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "ebb8a2df510c46649c3ab05050c5c8cb"
format_stamp: "Formatted at 2026-08-06 18:44:55 on dist-test-slave-fgb3"
server_key: "83c7959925458888b90225cd27745687"
server_key_iv: "4ae62a93053480fdfa7ceaeb98f0067e"
server_key_version: "encryptionkey@0"
I20260806 18:44:55.200793 6630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:44:55.203254 6647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:55.204541 6630 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:55.204663 6630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "ebb8a2df510c46649c3ab05050c5c8cb"
format_stamp: "Formatted at 2026-08-06 18:44:55 on dist-test-slave-fgb3"
server_key: "83c7959925458888b90225cd27745687"
server_key_iv: "4ae62a93053480fdfa7ceaeb98f0067e"
server_key_version: "encryptionkey@0"
I20260806 18:44:55.204762 6630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:55.216934 6630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:55.220177 6630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:55.220364 6630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:55.220948 6630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:55.221757 6630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:55.221805 6630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:55.221858 6630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:55.221887 6630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:55.230247 6630 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:40187
I20260806 18:44:55.230373 6806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:40187 every 8 connection(s)
I20260806 18:44:55.231153 6630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:44:55.238070 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 6630
I20260806 18:44:55.238179 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:44:55.238519 2005 external_mini_cluster.cc:1428] Setting key a9edbfb30f6fa2a293280fe70d5e7cad
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[6283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:55.245105 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:55.232995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:37321 (local address 127.1.245.126:38757)
0806 18:44:55.233136 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:55.233140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:55.233947 (+ 807us) server_negotiation.cc:407] Connection header received
0806 18:44:55.235359 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:55.235362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:55.235410 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:55.235487 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:55.236753 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.237290 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.238613 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.238936 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.239023 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:55.241706 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:55.241710 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:55.241711 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:55.241744 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:55.243078 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:55.243573 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:55.243576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:55.243577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:55.243651 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:55.243995 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:55.243999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:55.244001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:55.244183 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:55.244269 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:55.244803 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 18:44:55.244939 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:44:55.246002 6809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:55.233324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38757 (local address 127.1.245.66:37321)
0806 18:44:55.233779 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:55.233821 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:55.235145 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:55.235643 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:55.235650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:55.236016 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:55.236621 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:55.236635 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.237409 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.237411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:55.238493 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:55.238503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.239121 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.239126 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:55.239363 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:55.240049 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:55.240071 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:55.241532 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:55.243193 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:55.243199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:55.243215 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:55.243474 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:55.243738 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:55.243741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:55.243743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:55.243902 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:55.244294 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:55.244301 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:55.244552 (+ 251us) client_negotiation.cc:769] Sending connection context
0806 18:44:55.244729 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 18:44:55.245071 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":111,"threads_started":1}
I20260806 18:44:55.247265 6807 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38757
I20260806 18:44:55.247534 6807 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:55.248095 6807 heartbeater.cc:511] Master 127.1.245.126:38757 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:44:55.249224 6335 ts_manager.cc:194] Registered new tserver with Master: ebb8a2df510c46649c3ab05050c5c8cb (127.1.245.66:40187)
I20260806 18:44:55.249818 6335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:37321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:44:55.269023 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38757
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:55.384268 6812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:55.384545 6812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:55.384588 6812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:55.388628 6812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:55.388695 6812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:55.388756 6812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:44:55.394186 6812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38757
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:55.395434 6812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:55.396551 6812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:55.406266 6818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:55.406358 6821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:55.406286 6819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:55.406570 6812 server_base.cc:1034] running on GCE node
I20260806 18:44:55.407168 6812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:55.407853 6812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:55.409051 6812 hybrid_clock.cc:648] HybridClock initialized: now 1786041895409033 us; error 34 us; skew 500 ppm
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[6283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:55.411803 6812 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:44:55.412691 6812 webserver.cc:460] Webserver started at http://127.1.245.67:43321/ using document root <none> and password file <none>
I20260806 18:44:55.413221 6812 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:55.413280 6812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:55.413453 6812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:55.415551 6812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "a0c22ab2c7034c04bcfd95c5eb648128"
format_stamp: "Formatted at 2026-08-06 18:44:55 on dist-test-slave-fgb3"
server_key: "9cabade16c2bf466fdb1f72c8ed42c3e"
server_key_iv: "357539c8035a688ef0d7eac76b2c9dbf"
server_key_version: "encryptionkey@0"
I20260806 18:44:55.416097 6812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "a0c22ab2c7034c04bcfd95c5eb648128"
format_stamp: "Formatted at 2026-08-06 18:44:55 on dist-test-slave-fgb3"
server_key: "9cabade16c2bf466fdb1f72c8ed42c3e"
server_key_iv: "357539c8035a688ef0d7eac76b2c9dbf"
server_key_version: "encryptionkey@0"
I20260806 18:44:55.419976 6812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:44:55.422622 6828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:55.423920 6812 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:44:55.424032 6812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "a0c22ab2c7034c04bcfd95c5eb648128"
format_stamp: "Formatted at 2026-08-06 18:44:55 on dist-test-slave-fgb3"
server_key: "9cabade16c2bf466fdb1f72c8ed42c3e"
server_key_iv: "357539c8035a688ef0d7eac76b2c9dbf"
server_key_version: "encryptionkey@0"
I20260806 18:44:55.424144 6812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:55.439604 6812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:55.442762 6812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:55.442957 6812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:55.443575 6812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:55.444435 6812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:55.444483 6812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:55.444526 6812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:55.444554 6812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:55.453570 6812 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:45891
I20260806 18:44:55.453655 6990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:45891 every 8 connection(s)
I20260806 18:44:55.454689 6812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:44:55.456298 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 6812
I20260806 18:44:55.456430 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:44:55.456754 2005 external_mini_cluster.cc:1428] Setting key b68187cb4601de4cd79bdd06a4fe0614
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[6283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041895, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:55.468940 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:55.456692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:47827 (local address 127.1.245.126:38757)
0806 18:44:55.456844 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:55.456849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:55.457853 (+ 1004us) server_negotiation.cc:407] Connection header received
0806 18:44:55.459409 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:55.459420 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:55.459477 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:55.459553 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:55.460754 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.461293 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.462454 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.462674 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.462764 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:55.465375 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:55.465379 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:55.465381 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:55.465409 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:55.466876 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:55.467391 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:55.467394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:55.467396 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:55.467477 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:55.467816 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:55.467820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:55.467822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:55.468013 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:55.468103 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:55.468639 (+ 536us) server_negotiation.cc:299] Negotiation successful
0806 18:44:55.468773 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:44:55.469892 6993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:55.457208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38757 (local address 127.1.245.67:47827)
0806 18:44:55.457703 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:55.457757 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:55.459187 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:55.459744 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:55.459751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:55.460116 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:55.460603 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:55.460617 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.461419 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.461422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:55.462295 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:55.462305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.462858 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.462863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:55.463022 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:55.463608 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:55.463628 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:55.465189 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:55.466994 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:55.467000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:55.467016 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:55.467286 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:55.467567 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:55.467570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:55.467572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:55.467712 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:55.468119 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:55.468126 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:55.468393 (+ 267us) client_negotiation.cc:769] Sending connection context
0806 18:44:55.468572 (+ 179us) client_negotiation.cc:240] Negotiation successful
0806 18:44:55.468945 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":112,"threads_started":1}
I20260806 18:44:55.471107 6991 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38757
I20260806 18:44:55.471362 6991 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:55.471861 6991 heartbeater.cc:511] Master 127.1.245.126:38757 requested a full tablet report, sending...
I20260806 18:44:55.472872 6334 ts_manager.cc:194] Registered new tserver with Master: a0c22ab2c7034c04bcfd95c5eb648128 (127.1.245.67:45891)
I20260806 18:44:55.473428 6334 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:47827
I20260806 18:44:55.483145 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[6283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041895, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[6283](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041895, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:55.510375 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:55.501270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.1.245.126:38757)
0806 18:44:55.501427 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:55.501432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:55.501579 (+ 147us) server_negotiation.cc:407] Connection header received
0806 18:44:55.501808 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:55.501811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:55.501861 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:55.501938 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:55.502746 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.503321 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.504165 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:55.504422 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:55.504503 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:55.507082 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:55.507085 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:55.507087 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:55.507112 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:55.508654 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:55.509082 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:55.509085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:55.509086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:55.509174 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:55.509469 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:55.509472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:55.509473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:55.509708 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:55.509817 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:55.510077 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 18:44:55.510176 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 18:44:55.510742 6334 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39964
I20260806 18:44:55.511575 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 6424
I20260806 18:44:55.533250 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 6630
I20260806 18:44:55.542205 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 6812
I20260806 18:44:55.557446 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 6299
2026-08-06T18:44:55Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2281 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[7005](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[7005](info): set up 2 sockets
Aug 06 18:44:55 dist-test-slave-fgb3 krb5kdc[7005](info): commencing operation
krb5kdc: starting...
W20260806 18:44:56.685539 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:56 dist-test-slave-fgb3 krb5kdc[7005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:56.809630 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33353
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:33353
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:56.922200 7019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:56.922549 7019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:56.922591 7019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:56.926759 7019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:56.926834 7019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:56.926854 7019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:56.926869 7019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:56.926883 7019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:56.932412 7019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:33353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:56.933610 7019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:56.934727 7019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:56.943275 7026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:56.943424 7025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:56.943552 7019 server_base.cc:1034] running on GCE node
W20260806 18:44:56.943333 7030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:56.944231 7019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:56.945338 7019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:56.946506 7019 hybrid_clock.cc:648] HybridClock initialized: now 1786041896946480 us; error 35 us; skew 500 ppm
Aug 06 18:44:56 dist-test-slave-fgb3 krb5kdc[7005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041896, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:56.952873 7019 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:56.953930 7019 webserver.cc:460] Webserver started at http://127.1.245.126:44685/ using document root <none> and password file <none>
I20260806 18:44:56.954558 7019 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:56.954627 7019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:56.954804 7019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:56.956971 7019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "0df06e034bbf4d95a226482281e6e1ef"
format_stamp: "Formatted at 2026-08-06 18:44:56 on dist-test-slave-fgb3"
server_key: "45fc226c0b62c1377a62559bdfab5021"
server_key_iv: "877504002fa9ee647be65dd5308477fa"
server_key_version: "encryptionkey@0"
I20260806 18:44:56.957532 7019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "0df06e034bbf4d95a226482281e6e1ef"
format_stamp: "Formatted at 2026-08-06 18:44:56 on dist-test-slave-fgb3"
server_key: "45fc226c0b62c1377a62559bdfab5021"
server_key_iv: "877504002fa9ee647be65dd5308477fa"
server_key_version: "encryptionkey@0"
I20260806 18:44:56.961428 7019 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:44:56.964154 7042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:56.965551 7019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:44:56.965687 7019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "0df06e034bbf4d95a226482281e6e1ef"
format_stamp: "Formatted at 2026-08-06 18:44:56 on dist-test-slave-fgb3"
server_key: "45fc226c0b62c1377a62559bdfab5021"
server_key_iv: "877504002fa9ee647be65dd5308477fa"
server_key_version: "encryptionkey@0"
I20260806 18:44:56.965811 7019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:56.978920 7019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:56.982234 7019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:56.982470 7019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:56.989938 7019 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:33353
I20260806 18:44:56.989979 7122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:33353 every 8 connection(s)
I20260806 18:44:56.990972 7019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:56.993969 7124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:56.996404 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 7019
I20260806 18:44:56.996501 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:56.996757 2005 external_mini_cluster.cc:1428] Setting key 6fd608462148eb1d50487fb1f5817a0b
I20260806 18:44:57.000283 7124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef: Bootstrap starting.
I20260806 18:44:57.002688 7124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:57.003643 7124 log.cc:824] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:57.005769 7124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef: No bootstrap required, opened a new log
I20260806 18:44:57.009315 7124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0df06e034bbf4d95a226482281e6e1ef" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33353 } }
I20260806 18:44:57.009603 7124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:57.009649 7124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0df06e034bbf4d95a226482281e6e1ef, State: Initialized, Role: FOLLOWER
I20260806 18:44:57.010131 7124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0df06e034bbf4d95a226482281e6e1ef" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33353 } }
I20260806 18:44:57.010303 7124 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:57.010412 7124 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:57.010509 7124 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:57.011462 7124 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0df06e034bbf4d95a226482281e6e1ef" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33353 } }
I20260806 18:44:57.011510 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:56.998239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53796 (local address 127.1.245.126:33353)
0806 18:44:56.998763 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:56.998778 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:56.998815 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:44:56.999481 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:56.999499 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:56.999832 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:57.000154 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:57.000973 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.001980 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.002866 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.003588 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.003771 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.006355 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:57.006363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:57.006384 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:57.006418 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:57.008535 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.009046 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.009050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.009055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.009165 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.009438 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.009440 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.009441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.009886 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:57.010071 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:57.010394 (+ 323us) server_negotiation.cc:299] Negotiation successful
0806 18:44:57.010726 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":159,"threads_started":1}
I20260806 18:44:57.011806 7124 leader_election.cc:302] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0df06e034bbf4d95a226482281e6e1ef; no voters:
I20260806 18:44:57.012120 7124 leader_election.cc:288] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:57.012215 7133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:57.012384 7133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [term 1 LEADER]: Becoming Leader. State: Replica: 0df06e034bbf4d95a226482281e6e1ef, State: Running, Role: LEADER
I20260806 18:44:57.013453 7124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:57.013782 7133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0df06e034bbf4d95a226482281e6e1ef" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33353 } }
I20260806 18:44:57.015506 7136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0df06e034bbf4d95a226482281e6e1ef. Latest consensus state: current_term: 1 leader_uuid: "0df06e034bbf4d95a226482281e6e1ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0df06e034bbf4d95a226482281e6e1ef" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33353 } } }
I20260806 18:44:57.015652 7136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:57.015501 7135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0df06e034bbf4d95a226482281e6e1ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0df06e034bbf4d95a226482281e6e1ef" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33353 } } }
I20260806 18:44:57.016570 7135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef [sys.catalog]: This master's current role is: LEADER
W20260806 18:44:57.018612 7155 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:44:57.018687 7155 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:44:57.018745 7156 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:57.019503 7156 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:57.025331 7156 catalog_manager.cc:1347] Generated new cluster ID: 6819e4447747420e8c77d99ae584558e
I20260806 18:44:57.025415 7156 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:57.087116 7156 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:57.088110 7156 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:57.093560 7156 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0df06e034bbf4d95a226482281e6e1ef: Generated new TSK 0
I20260806 18:44:57.094363 7156 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:44:57.138276 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33353
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:57.252250 7165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:57.252508 7165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:57.252550 7165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:57.256654 7165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:57.256724 7165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:57.256789 7165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:44:57.262099 7165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:57.263213 7165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:57.264247 7165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:57.273826 7175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:57.273797 7178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:57.273778 7176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:57.274240 7165 server_base.cc:1034] running on GCE node
I20260806 18:44:57.274667 7165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:57.275290 7165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:57.276472 7165 hybrid_clock.cc:648] HybridClock initialized: now 1786041897276455 us; error 30 us; skew 500 ppm
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:57.279305 7165 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:44:57.280215 7165 webserver.cc:460] Webserver started at http://127.1.245.65:40637/ using document root <none> and password file <none>
I20260806 18:44:57.280777 7165 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:57.280843 7165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:57.281037 7165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:57.283139 7165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "4e8a218f9e6a40628cd88d54b4912d24"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "b6cd63dfd50b48bebcfefcd9daf61432"
server_key_iv: "3376b642c502efa271bba9d04e502fa2"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.283671 7165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "4e8a218f9e6a40628cd88d54b4912d24"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "b6cd63dfd50b48bebcfefcd9daf61432"
server_key_iv: "3376b642c502efa271bba9d04e502fa2"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.287493 7165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260806 18:44:57.290036 7187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.291378 7165 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:57.291498 7165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "4e8a218f9e6a40628cd88d54b4912d24"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "b6cd63dfd50b48bebcfefcd9daf61432"
server_key_iv: "3376b642c502efa271bba9d04e502fa2"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.291600 7165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:57.337072 7165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:57.340528 7165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:57.340830 7165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:57.341512 7165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:57.342511 7165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:57.342566 7165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.342624 7165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:57.342650 7165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.352140 7165 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:40581
I20260806 18:44:57.352216 7360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:40581 every 8 connection(s)
I20260806 18:44:57.353201 7165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:44:57.355825 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 7165
I20260806 18:44:57.355939 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:44:57.356336 2005 external_mini_cluster.cc:1428] Setting key 9ce749f5ff21629496d4d6f3f0dc3e18
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:44:57.369465 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:57.355288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:37069 (local address 127.1.245.126:33353)
0806 18:44:57.355512 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:57.355517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:57.356538 (+ 1021us) server_negotiation.cc:407] Connection header received
0806 18:44:57.358258 (+ 1720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:57.358261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:57.358308 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:57.358443 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:57.360051 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.360843 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.362135 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.362531 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.362646 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.365315 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:57.365319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:57.365321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:57.365357 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:57.367019 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.367723 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.367726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.367728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.367837 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.368293 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.368296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.368297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.368509 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:57.368591 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:57.369204 (+ 613us) server_negotiation.cc:299] Negotiation successful
0806 18:44:57.369329 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:44:57.370931 7364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:57.355686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33353 (local address 127.1.245.65:37069)
0806 18:44:57.356344 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:57.356410 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:57.358022 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:57.358641 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:57.358650 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:57.359149 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:57.359863 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:57.359887 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.361052 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.361055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:57.361994 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:57.362005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.362697 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.362701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:57.362894 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.363558 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:57.363576 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:57.365112 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:57.367156 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:57.367162 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:57.367178 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:57.367610 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:57.367956 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:57.367959 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:57.367960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:57.368158 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:57.368615 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:57.368621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:57.368948 (+ 327us) client_negotiation.cc:769] Sending connection context
0806 18:44:57.369123 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 18:44:57.369585 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":148,"threads_started":1}
I20260806 18:44:57.372303 7361 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33353
I20260806 18:44:57.372609 7361 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:57.373239 7361 heartbeater.cc:511] Master 127.1.245.126:33353 requested a full tablet report, sending...
I20260806 18:44:57.375200 7071 ts_manager.cc:194] Registered new tserver with Master: 4e8a218f9e6a40628cd88d54b4912d24 (127.1.245.65:40581)
I20260806 18:44:57.376891 7071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:37069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:44:57.389958 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33353
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:57.505657 7368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:57.505927 7368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:57.505970 7368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:57.509903 7368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:57.509958 7368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:57.510015 7368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:44:57.515286 7368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:57.516397 7368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:57.517352 7368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:57.526769 7376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:57.526861 7368 server_base.cc:1034] running on GCE node
W20260806 18:44:57.526814 7374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:57.526825 7380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:57.527357 7368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:57.527974 7368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:57.529138 7368 hybrid_clock.cc:648] HybridClock initialized: now 1786041897529119 us; error 32 us; skew 500 ppm
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:57.531773 7368 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:44:57.532630 7368 webserver.cc:460] Webserver started at http://127.1.245.66:45105/ using document root <none> and password file <none>
I20260806 18:44:57.533155 7368 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:57.533224 7368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:57.533411 7368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:57.535396 7368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "b95a8e6b42944160923327f1bcbbbf3a"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "01c5327444f37c8e8317ac425da63d04"
server_key_iv: "a7c303b7571efb4f2699b400aef6078b"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.535912 7368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "b95a8e6b42944160923327f1bcbbbf3a"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "01c5327444f37c8e8317ac425da63d04"
server_key_iv: "a7c303b7571efb4f2699b400aef6078b"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.539436 7368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:44:57.541759 7392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.543141 7368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:57.543259 7368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "b95a8e6b42944160923327f1bcbbbf3a"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "01c5327444f37c8e8317ac425da63d04"
server_key_iv: "a7c303b7571efb4f2699b400aef6078b"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.543365 7368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:57.560489 7368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:57.563858 7368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:57.564061 7368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:57.564728 7368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:57.565675 7368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:57.565739 7368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.565809 7368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:57.565840 7368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.580420 7368 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:38973
I20260806 18:44:57.580435 7564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:38973 every 8 connection(s)
I20260806 18:44:57.581446 7368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:44:57.587360 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 7368
I20260806 18:44:57.587494 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:44:57.587836 2005 external_mini_cluster.cc:1428] Setting key 2bef185e6ed956a4a93d8668778c172e
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:57.597611 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:57.583713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:59285 (local address 127.1.245.126:33353)
0806 18:44:57.583876 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:57.583881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:57.584818 (+ 937us) server_negotiation.cc:407] Connection header received
0806 18:44:57.586537 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:57.586541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:57.586605 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:57.586789 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:57.588246 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.588866 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.590428 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.590862 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.590984 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.594006 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:57.594011 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:57.594014 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:57.594047 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:57.595554 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.596099 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.596102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.596104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.596187 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.596506 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.596510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.596512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.596695 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:57.596780 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:57.597315 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 18:44:57.597444 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:44:57.598587 7569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:57.584118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33353 (local address 127.1.245.66:59285)
0806 18:44:57.584634 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:57.584689 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:57.586269 (+ 1580us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:57.586957 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:57.586966 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:57.587516 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:57.588103 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:57.588120 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.588995 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.588999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:57.590269 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:57.590281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.591042 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.591046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:57.591253 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.591831 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:57.591849 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:57.593815 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:57.595665 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:57.595671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:57.595688 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:57.596005 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:57.596271 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:57.596274 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:57.596276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:57.596412 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:57.596798 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:57.596805 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:57.597060 (+ 255us) client_negotiation.cc:769] Sending connection context
0806 18:44:57.597256 (+ 196us) client_negotiation.cc:240] Negotiation successful
0806 18:44:57.597582 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":142,"threads_started":1}
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:44:57.599798 7565 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33353
I20260806 18:44:57.600059 7565 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:57.600608 7565 heartbeater.cc:511] Master 127.1.245.126:33353 requested a full tablet report, sending...
I20260806 18:44:57.601696 7071 ts_manager.cc:194] Registered new tserver with Master: b95a8e6b42944160923327f1bcbbbf3a (127.1.245.66:38973)
I20260806 18:44:57.602419 7071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:59285
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:44:57.618072 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33353
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:34351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:57.731928 7572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:57.732196 7572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:57.732239 7572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:57.736379 7572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:57.736452 7572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:57.736521 7572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:44:57.741899 7572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:34351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33353
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:57.743072 7572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:57.744158 7572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:57.753928 7581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:57.753947 7584 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:57.753968 7578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:57.754359 7572 server_base.cc:1034] running on GCE node
I20260806 18:44:57.754740 7572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:57.755409 7572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:57.756585 7572 hybrid_clock.cc:648] HybridClock initialized: now 1786041897756553 us; error 42 us; skew 500 ppm
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:57.759367 7572 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:44:57.760195 7572 webserver.cc:460] Webserver started at http://127.1.245.67:44759/ using document root <none> and password file <none>
I20260806 18:44:57.760746 7572 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:57.760807 7572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:57.760982 7572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:57.763130 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "9b7388ce59c9431096786f01298cb182"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "d20016bd46b7a458ce26b16c927227ad"
server_key_iv: "7ffb9ce85cde13025a1906b9ae41885c"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.763651 7572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "9b7388ce59c9431096786f01298cb182"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "d20016bd46b7a458ce26b16c927227ad"
server_key_iv: "7ffb9ce85cde13025a1906b9ae41885c"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.767251 7572 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:44:57.769537 7598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.770819 7572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:57.770913 7572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "9b7388ce59c9431096786f01298cb182"
format_stamp: "Formatted at 2026-08-06 18:44:57 on dist-test-slave-fgb3"
server_key: "d20016bd46b7a458ce26b16c927227ad"
server_key_iv: "7ffb9ce85cde13025a1906b9ae41885c"
server_key_version: "encryptionkey@0"
I20260806 18:44:57.770991 7572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:57.795614 7572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:57.798933 7572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:57.799098 7572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:57.799690 7572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:57.800524 7572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:57.800572 7572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.800611 7572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:57.800631 7572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:57.808815 7572 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:38737
I20260806 18:44:57.808851 7780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:38737 every 8 connection(s)
I20260806 18:44:57.809702 7572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:44:57.815327 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 7572
I20260806 18:44:57.815454 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:44:57.815707 2005 external_mini_cluster.cc:1428] Setting key f82a3c976c9d8e72e40c9b46b8580d87
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:57.823568 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:57.811698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:54939 (local address 127.1.245.126:33353)
0806 18:44:57.811841 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:57.811845 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:57.812684 (+ 839us) server_negotiation.cc:407] Connection header received
0806 18:44:57.814152 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:57.814155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:57.814214 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:57.814287 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:57.815540 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.816128 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.817564 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.817849 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.817934 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.820316 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:57.820319 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:57.820321 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:57.820347 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:57.821642 (+ 1295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.822116 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.822119 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.822120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.822188 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.822517 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.822520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.822522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.822713 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:57.822795 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:57.823330 (+ 535us) server_negotiation.cc:299] Negotiation successful
0806 18:44:57.823438 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:44:57.824366 7786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:57.812068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33353 (local address 127.1.245.67:54939)
0806 18:44:57.812525 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:57.812567 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:57.813941 (+ 1374us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:57.814511 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:57.814518 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:57.814863 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:57.815379 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:57.815397 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.816261 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.816264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:57.817446 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:57.817457 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.818006 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.818009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:57.818151 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.818713 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:57.818730 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:57.820160 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:57.821754 (+ 1594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:57.821759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:57.821772 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:57.822039 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:57.822252 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:57.822254 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:57.822256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:57.822427 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:57.822798 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:57.822802 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:57.823058 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 18:44:57.823212 (+ 154us) client_negotiation.cc:240] Negotiation successful
0806 18:44:57.823476 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":118,"threads_started":1}
I20260806 18:44:57.825560 7782 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33353
I20260806 18:44:57.825802 7782 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:57.826290 7782 heartbeater.cc:511] Master 127.1.245.126:33353 requested a full tablet report, sending...
I20260806 18:44:57.827328 7071 ts_manager.cc:194] Registered new tserver with Master: 9b7388ce59c9431096786f01298cb182 (127.1.245.67:38737)
I20260806 18:44:57.827924 7071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:54939
I20260806 18:44:57.829471 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:44:57.855398 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:57.845870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36158 (local address 127.1.245.65:40581)
0806 18:44:57.846164 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:57.846180 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:57.846204 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:44:57.846258 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:57.846263 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:57.846520 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:57.846643 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:57.847419 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.848257 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.849089 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.849455 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.849534 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.851926 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:57.851937 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:57.851962 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:57.851987 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:57.853509 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.853916 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.853921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.853922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.854011 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.854295 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.854298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.854300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.854599 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:57.854835 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:57.855038 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 18:44:57.855197 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":82,"threads_started":1}
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7005](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:57.866274 7131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:57.858416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53812 (local address 127.1.245.126:33353)
0806 18:44:57.858549 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:57.858554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:57.858585 (+ 31us) server_negotiation.cc:407] Connection header received
0806 18:44:57.858729 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:57.858732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:57.858783 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:57.858850 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:57.859463 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.859986 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.860735 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:57.860962 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:57.861051 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:57.863362 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:57.863366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:57.863369 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:57.863396 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:57.864646 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.865056 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.865059 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.865061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.865139 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:57.865397 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:57.865401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:57.865402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:57.865624 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:57.865747 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:57.866006 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 18:44:57.866116 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260806 18:44:57.866726 7071 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53812
I20260806 18:44:57.867434 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 7165
I20260806 18:44:57.877700 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 7368
I20260806 18:44:57.888918 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 7572
I20260806 18:44:57.900564 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 7019
2026-08-06T18:44:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2340 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7796](info): set up 2 sockets
Aug 06 18:44:57 dist-test-slave-fgb3 krb5kdc[7796](info): commencing operation
krb5kdc: starting...
W20260806 18:44:59.039655 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:44:59 dist-test-slave-fgb3 krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:44:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:44:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:44:59.165000 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:41133
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:41133
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:59.277727 7812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:59.278031 7812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:59.278075 7812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:59.282083 7812 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:44:59.282146 7812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:59.282164 7812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:59.282179 7812 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:44:59.282193 7812 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:44:59.287575 7812 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:41133
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:41133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:59.288724 7812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:59.289774 7812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:59.298506 7820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:59.298575 7812 server_base.cc:1034] running on GCE node
W20260806 18:44:59.298521 7818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:59.298521 7823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:59.299353 7812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:59.300436 7812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:59.301617 7812 hybrid_clock.cc:648] HybridClock initialized: now 1786041899301593 us; error 36 us; skew 500 ppm
Aug 06 18:44:59 dist-test-slave-fgb3 krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:59.304440 7812 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:44:59.305184 7812 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:59.305255 7812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:59.305472 7812 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:59.307585 7812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "a0408320ac064a7391f37701925a965b"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "d974896560aed4fbb3ea569a256d0c0a"
server_key_iv: "7df7647ac33a6440335ad50069e87a5b"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.308123 7812 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "a0408320ac064a7391f37701925a965b"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "d974896560aed4fbb3ea569a256d0c0a"
server_key_iv: "7df7647ac33a6440335ad50069e87a5b"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.311985 7812 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:44:59.314518 7829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:59.315943 7812 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:44:59.316076 7812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "a0408320ac064a7391f37701925a965b"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "d974896560aed4fbb3ea569a256d0c0a"
server_key_iv: "7df7647ac33a6440335ad50069e87a5b"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.316200 7812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:59.332072 7812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:59.335209 7812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:59.335383 7812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:59.342741 7812 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:41133
I20260806 18:44:59.343003 7906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:41133 every 8 connection(s)
I20260806 18:44:59.343457 7812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:44:59.346264 7907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:44:59.351661 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 7812
I20260806 18:44:59.351794 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:44:59.351867 7907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b: Bootstrap starting.
I20260806 18:44:59.352056 2005 external_mini_cluster.cc:1428] Setting key f35ea34f4a84fed199c07cb00f472620
I20260806 18:44:59.354342 7907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b: Neither blocks nor log segments found. Creating new log.
I20260806 18:44:59.355093 7907 log.cc:824] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:44:59.356971 7907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b: No bootstrap required, opened a new log
Aug 06 18:44:59 dist-test-slave-fgb3 krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041899, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:59.361330 7907 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0408320ac064a7391f37701925a965b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41133 } }
I20260806 18:44:59.361615 7907 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:44:59.361680 7907 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0408320ac064a7391f37701925a965b, State: Initialized, Role: FOLLOWER
I20260806 18:44:59.362087 7907 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0408320ac064a7391f37701925a965b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41133 } }
I20260806 18:44:59.362205 7907 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:44:59.362272 7907 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:44:59.362391 7907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:44:59.363224 7907 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0408320ac064a7391f37701925a965b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41133 } }
I20260806 18:44:59.363546 7907 leader_election.cc:302] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0408320ac064a7391f37701925a965b; no voters:
I20260806 18:44:59.363893 7907 leader_election.cc:288] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:44:59.364060 7914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:44:59.364321 7914 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [term 1 LEADER]: Becoming Leader. State: Replica: a0408320ac064a7391f37701925a965b, State: Running, Role: LEADER
I20260806 18:44:59.364643 7914 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0408320ac064a7391f37701925a965b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41133 } }
I20260806 18:44:59.365108 7907 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:44:59.365861 7914 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0408320ac064a7391f37701925a965b. Latest consensus state: current_term: 1 leader_uuid: "a0408320ac064a7391f37701925a965b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0408320ac064a7391f37701925a965b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41133 } } }
I20260806 18:44:59.365972 7914 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:59.366459 7916 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0408320ac064a7391f37701925a965b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0408320ac064a7391f37701925a965b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41133 } } }
I20260806 18:44:59.366626 7916 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b [sys.catalog]: This master's current role is: LEADER
I20260806 18:44:59.366858 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:59.353352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44252 (local address 127.1.245.126:41133)
0806 18:44:59.353770 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:59.353787 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:59.353833 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:44:59.354522 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:59.354542 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:59.354877 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:59.355144 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:59.355956 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.356943 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.358232 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.358764 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.358890 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:59.361684 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:59.361691 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:59.361709 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:59.361745 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:59.363501 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:59.363992 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:59.363999 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:59.364005 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:59.364159 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:59.364510 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:59.364513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:59.364514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:59.365036 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:59.365225 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:59.365533 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 18:44:59.365852 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":122,"threads_started":1}
I20260806 18:44:59.367161 7924 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:44:59.370370 7924 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:44:59.375980 7924 catalog_manager.cc:1347] Generated new cluster ID: 790420fe51d34d30b8fe355f20259476
I20260806 18:44:59.376067 7924 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:44:59.397442 7924 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:44:59.398651 7924 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:44:59.404103 7924 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b: Generated new TSK 0
I20260806 18:44:59.404789 7924 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:44:59.433815 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41133
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:59.546729 7945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:59.547011 7945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:59.547073 7945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:59.551105 7945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:59.551165 7945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:59.551246 7945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:44:59.556548 7945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7945
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:59.557622 7945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:59.558789 7945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:44:59.568300 7945 server_base.cc:1034] running on GCE node
W20260806 18:44:59.568189 7953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:59.568189 7952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:59.568275 7957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:59.568883 7945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:59.569475 7945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:59.570628 7945 hybrid_clock.cc:648] HybridClock initialized: now 1786041899570606 us; error 31 us; skew 500 ppm
Aug 06 18:44:59 dist-test-slave-fgb3 krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:59.573268 7945 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:44:59.573957 7945 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:59.574024 7945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:59.574206 7945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:59.576206 7945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "be357c1e12a24fd49156fa0b09058bff"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "164bce954b74aece0ce636a87dba8e49"
server_key_iv: "0e7a66a2474b1b39883bf3aca3f6d8de"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.576735 7945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "be357c1e12a24fd49156fa0b09058bff"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "164bce954b74aece0ce636a87dba8e49"
server_key_iv: "0e7a66a2474b1b39883bf3aca3f6d8de"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.580334 7945 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:44:59.582647 7966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:59.583907 7945 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:44:59.584013 7945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "be357c1e12a24fd49156fa0b09058bff"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "164bce954b74aece0ce636a87dba8e49"
server_key_iv: "0e7a66a2474b1b39883bf3aca3f6d8de"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.584110 7945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:59.605577 7945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:59.608789 7945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:59.608968 7945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:59.609534 7945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:59.610409 7945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:59.610458 7945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:59.610512 7945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:59.610538 7945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:59.618955 7945 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:46615
I20260806 18:44:59.619005 8117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:46615 every 8 connection(s)
I20260806 18:44:59.619530 7945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:44:59.620640 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 7945
I20260806 18:44:59.620743 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:44:59.620998 2005 external_mini_cluster.cc:1428] Setting key 3c61e4bf615e84e426cc1c825790a463
Aug 06 18:44:59 dist-test-slave-fgb3 krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:44:59.634084 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:59.621326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:50819 (local address 127.1.245.126:41133)
0806 18:44:59.621464 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:59.621469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:59.622560 (+ 1091us) server_negotiation.cc:407] Connection header received
0806 18:44:59.624266 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:59.624269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:59.624336 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:59.624441 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:59.625590 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.626143 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.627383 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.627617 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.627707 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:59.630457 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:59.630460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:59.630462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:59.630488 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:59.632043 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:59.632527 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:59.632529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:59.632530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:59.632602 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:59.633024 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:59.633028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:59.633029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:59.633214 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:59.633327 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:59.633842 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 18:44:59.633948 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:44:59.635004 8123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:59.621786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41133 (local address 127.1.245.65:50819)
0806 18:44:59.622418 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:59.622471 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:59.624016 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:59.624617 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:59.624624 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:59.624966 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:59.625442 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:59.625455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.626306 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.626335 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:59.627255 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:59.627275 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.627755 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.627760 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:59.627934 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:59.628457 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:59.628473 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:59.630263 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:59.632154 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:59.632159 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:59.632171 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:59.632430 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:59.632707 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:59.632710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:59.632711 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:59.632923 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:59.633348 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:59.633354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:59.633594 (+ 240us) client_negotiation.cc:769] Sending connection context
0806 18:44:59.633767 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 18:44:59.634055 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":151,"threads_started":1}
I20260806 18:44:59.636211 8120 heartbeater.cc:347] Connected to a master server at 127.1.245.126:41133
I20260806 18:44:59.636469 8120 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:59.636907 8120 heartbeater.cc:511] Master 127.1.245.126:41133 requested a full tablet report, sending...
I20260806 18:44:59.638430 7852 ts_manager.cc:194] Registered new tserver with Master: be357c1e12a24fd49156fa0b09058bff (127.1.245.65:46615)
I20260806 18:44:59.639556 7852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:50819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:44:59.650835 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41133
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:44:59.761950 8130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:44:59.762228 8130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:44:59.762290 8130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:44:59.766402 8130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:44:59.766474 8130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:44:59.766561 8130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:44:59.771795 8130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:44:59.772832 8130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:44:59.773837 8130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:44:59.783033 8139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:59.783128 8135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:44:59.783070 8136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:44:59.783434 8130 server_base.cc:1034] running on GCE node
I20260806 18:44:59.783810 8130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:44:59.784444 8130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:44:59.785646 8130 hybrid_clock.cc:648] HybridClock initialized: now 1786041899785614 us; error 44 us; skew 500 ppm
Aug 06 18:44:59 dist-test-slave-fgb3 krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:44:59.788538 8130 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:44:59.789283 8130 fs_manager.cc:359] Metadata directory not provided
I20260806 18:44:59.789350 8130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:44:59.789567 8130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:44:59.791771 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "1093b85ebc8a49fca2b1707f38e7f2fd"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "09b9f1596e8602a27016ad56070e73b9"
server_key_iv: "5c49a4e483400baeef087f22d16a3dfc"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.792330 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "1093b85ebc8a49fca2b1707f38e7f2fd"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "09b9f1596e8602a27016ad56070e73b9"
server_key_iv: "5c49a4e483400baeef087f22d16a3dfc"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.796988 8130 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260806 18:44:59.799746 8145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:59.801054 8130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:44:59.801180 8130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "1093b85ebc8a49fca2b1707f38e7f2fd"
format_stamp: "Formatted at 2026-08-06 18:44:59 on dist-test-slave-fgb3"
server_key: "09b9f1596e8602a27016ad56070e73b9"
server_key_iv: "5c49a4e483400baeef087f22d16a3dfc"
server_key_version: "encryptionkey@0"
I20260806 18:44:59.801321 8130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:44:59.816237 8130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:44:59.820062 8130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:44:59.820281 8130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:44:59.820925 8130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:44:59.821872 8130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:44:59.821936 8130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:59.821996 8130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:44:59.822032 8130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:44:59.831014 8130 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:46389
I20260806 18:44:59.831038 8295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:46389 every 8 connection(s)
I20260806 18:44:59.831634 8130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:44:59.837072 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 8130
I20260806 18:44:59.837219 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:44:59.837564 2005 external_mini_cluster.cc:1428] Setting key 2393db7344ac28885a3c877c2d245993
Aug 06 18:44:59 dist-test-slave-fgb3 krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041899, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:44:59.846603 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:59.833469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:58475 (local address 127.1.245.126:41133)
0806 18:44:59.833603 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:44:59.833607 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:44:59.834555 (+ 948us) server_negotiation.cc:407] Connection header received
0806 18:44:59.835914 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:44:59.835918 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:44:59.835981 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:44:59.836060 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:44:59.837532 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.838302 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.839944 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.840275 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.840386 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:59.843086 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:44:59.843090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:44:59.843092 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:44:59.843116 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:44:59.844443 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:59.845002 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:59.845005 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:59.845006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:59.845093 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:44:59.845415 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:44:59.845418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:44:59.845420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:44:59.845604 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:44:59.845702 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:44:59.846257 (+ 555us) server_negotiation.cc:299] Negotiation successful
0806 18:44:59.846413 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:44:59.847415 8299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:44:59.833884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41133 (local address 127.1.245.66:58475)
0806 18:44:59.834400 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 18:44:59.834446 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:44:59.835701 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:44:59.836244 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:44:59.836251 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:44:59.836626 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:44:59.837356 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:59.837380 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.838463 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.838466 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:59.839655 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:44:59.839666 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:44:59.840426 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:44:59.840429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:44:59.840575 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:44:59.841163 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:44:59.841179 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:44:59.842919 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:44:59.844555 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:59.844562 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:59.844580 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:59.844907 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:59.845178 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:44:59.845181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:44:59.845183 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:44:59.845327 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:44:59.845719 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:44:59.845733 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:44:59.845996 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 18:44:59.846173 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 18:44:59.846551 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:44:59.848641 8297 heartbeater.cc:347] Connected to a master server at 127.1.245.126:41133
I20260806 18:44:59.848896 8297 heartbeater.cc:464] Registering TS with master...
I20260806 18:44:59.849356 8297 heartbeater.cc:511] Master 127.1.245.126:41133 requested a full tablet report, sending...
I20260806 18:44:59.850462 7852 ts_manager.cc:194] Registered new tserver with Master: 1093b85ebc8a49fca2b1707f38e7f2fd (127.1.245.66:46389)
I20260806 18:44:59.851008 7852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:58475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
I20260806 18:45:00.641870 8120 heartbeater.cc:503] Master 127.1.245.126:41133 was elected leader, sending a full tablet report...
I20260806 18:45:00.852996 8297 heartbeater.cc:503] Master 127.1.245.126:41133 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
W20260806 18:45:01.004642 2005 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.245.67: real 1.165s user 0.000s sys 0.004s
I20260806 18:45:01.005223 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41133
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:46797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:01.122234 8303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:01.122574 8303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:01.122629 8303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:01.126785 8303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:01.126864 8303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:01.126935 8303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:01.132346 8303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41133
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:01.133469 8303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:01.134539 8303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:01.144111 8313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:01.144157 8309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:01.144168 8310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:01.144503 8303 server_base.cc:1034] running on GCE node
I20260806 18:45:01.144930 8303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:01.145588 8303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:01.146775 8303 hybrid_clock.cc:648] HybridClock initialized: now 1786041901146753 us; error 32 us; skew 500 ppm
Aug 06 18:45:01 dist-test-slave-fgb3 krb5kdc[7796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:01.149468 8303 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:45:01.150206 8303 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:01.150278 8303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:01.150511 8303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:01.152634 8303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "deead86419f75710811c78f77106f7db"
server_key_iv: "ab3b6d7932a55b1a06da67f6aa43256c"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.153189 8303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "deead86419f75710811c78f77106f7db"
server_key_iv: "ab3b6d7932a55b1a06da67f6aa43256c"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.156966 8303 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:45:01.159507 8319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.160938 8303 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:45:01.161072 8303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "deead86419f75710811c78f77106f7db"
server_key_iv: "ab3b6d7932a55b1a06da67f6aa43256c"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.161193 8303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:01.180274 8303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:01.183439 8303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:01.183609 8303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:01.184198 8303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:01.185065 8303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:01.185122 8303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.185179 8303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:01.185210 8303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.193723 8303 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:34829
I20260806 18:45:01.193761 8486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:34829 every 8 connection(s)
I20260806 18:45:01.194350 8303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:01.202443 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 8303
I20260806 18:45:01.202557 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:01.202826 2005 external_mini_cluster.cc:1428] Setting key f4c0f24e33dd7d3aab3652dd5b2cddf1
Aug 06 18:45:01 dist-test-slave-fgb3 krb5kdc[7796](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041901, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:01.208806 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.196214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:57749 (local address 127.1.245.126:41133)
0806 18:45:01.196521 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.196525 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.197258 (+ 733us) server_negotiation.cc:407] Connection header received
0806 18:45:01.198678 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.198681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.198742 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.198840 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:01.200016 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.200650 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.201858 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.202092 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.202199 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.205154 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:01.205158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:01.205161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:01.205190 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:01.206736 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:01.207233 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:01.207237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:01.207239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:01.207332 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:01.207666 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:01.207670 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:01.207672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:01.207870 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:01.207961 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.208488 (+ 527us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.208622 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":107,"threads_started":1}
I20260806 18:45:01.209650 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.196619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41133 (local address 127.1.245.67:57749)
0806 18:45:01.197097 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.197139 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:01.198450 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:01.198988 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:01.198994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:01.199377 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:01.199871 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.199885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.200823 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.200826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.201709 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.201731 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.202247 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.202252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.202463 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.203098 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:01.203119 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:01.204982 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:01.206852 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:01.206858 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:01.206875 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:01.207136 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:01.207419 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:01.207422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:01.207424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:01.207576 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:01.207970 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:01.207978 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:01.208245 (+ 267us) client_negotiation.cc:769] Sending connection context
0806 18:45:01.208428 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 18:45:01.208773 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":114,"threads_started":1}
I20260806 18:45:01.211015 8487 heartbeater.cc:347] Connected to a master server at 127.1.245.126:41133
I20260806 18:45:01.211366 8487 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:01.211853 8487 heartbeater.cc:511] Master 127.1.245.126:41133 requested a full tablet report, sending...
I20260806 18:45:01.212978 7852 ts_manager.cc:194] Registered new tserver with Master: 12e525cbded747a2a0cd2c0ebd1c7d8d (127.1.245.67:34829)
I20260806 18:45:01.213564 7852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:57749
I20260806 18:45:01.217818 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:45:01.227166 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.219892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44266 (local address 127.1.245.126:41133)
0806 18:45:01.220012 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.220015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.220172 (+ 157us) server_negotiation.cc:407] Connection header received
0806 18:45:01.220263 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.220265 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.220311 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.220386 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:01.221091 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.221618 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.222559 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.222803 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.222889 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.224126 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:01.224129 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:01.224131 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:01.224155 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:01.225545 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:01.225927 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:01.225930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:01.225931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:01.226019 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:01.226279 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:01.226282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:01.226283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:01.226550 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:01.226668 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.226881 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.227040 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:45:01.230106 7852 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44266:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:45:01.233217 7852 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:01.247798 8507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.241956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:46389 (local address 127.0.0.1:42604)
0806 18:45:01.242629 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.242680 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:01.242816 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:01.243684 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:01.243690 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:01.243716 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:01.243883 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.243891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.245367 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.245370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.246600 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.246606 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.247408 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.247410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.247486 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.247516 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 18:45:01.247600 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:45:01.247683 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":65,"thread_start_us":110,"threads_started":1}
I20260806 18:45:01.247931 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.242116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.1.245.66:46389)
0806 18:45:01.243114 (+ 998us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.243122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.243147 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:45:01.243270 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.243276 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.243493 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.243684 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:01.244003 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.245240 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.246725 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.247299 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.247399 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.247510 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.247662 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.247775 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":886,"thread_start_us":105,"threads_started":1}
I20260806 18:45:01.248883 8505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.242012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:46615 (local address 127.0.0.1:44150)
0806 18:45:01.242706 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.242720 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:01.242819 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:01.243801 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:01.243804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:01.243829 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:01.243918 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.243924 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.245092 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.245096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.246589 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.246600 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.248530 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.248534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.248603 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.248622 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:45:01.248672 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 18:45:01.248752 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":632,"mutex_wait_us":94,"thread_start_us":95,"threads_started":1}
I20260806 18:45:01.248987 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.242564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44150 (local address 127.1.245.65:46615)
0806 18:45:01.243364 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.243370 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.243391 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:45:01.243467 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.243472 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.243663 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.244106 (+ 443us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:01.244250 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.244986 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.247568 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.248385 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.248505 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.248645 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.248729 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.248823 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":93,"threads_started":1}
I20260806 18:45:01.250481 8504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.242232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:34829 (local address 127.0.0.1:43194)
0806 18:45:01.243454 (+ 1222us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.243469 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:01.243565 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:01.244062 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:01.244065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:01.244082 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:01.244170 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.244176 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.245473 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.245478 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.246594 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.246604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.250008 (+ 3404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.250012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.250114 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.250141 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:45:01.250235 (+ 94us) client_negotiation.cc:240] Negotiation successful
0806 18:45:01.250335 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1165,"mutex_wait_us":37,"thread_start_us":116,"threads_started":1}
I20260806 18:45:01.250778 8511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.242920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43194 (local address 127.1.245.67:34829)
0806 18:45:01.243255 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.243267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.243665 (+ 398us) server_negotiation.cc:407] Connection header received
0806 18:45:01.243759 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.243765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.243921 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.244097 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:01.244258 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.245340 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.249161 (+ 3821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.249873 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.249978 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.250116 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.250281 (+ 165us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.250426 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":144,"threads_started":1}
I20260806 18:45:01.251065 8212 tablet_service.cc:1467] Processing CreateTablet for tablet 8f648b1eecab42f68a21d52e1ce9cc89 (DEFAULT_TABLE table=test-table [id=278c82c0abab4fa7908bb0671e94f4b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:01.252043 8212 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f648b1eecab42f68a21d52e1ce9cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:01.252525 8030 tablet_service.cc:1467] Processing CreateTablet for tablet 8f648b1eecab42f68a21d52e1ce9cc89 (DEFAULT_TABLE table=test-table [id=278c82c0abab4fa7908bb0671e94f4b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:01.253495 8030 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f648b1eecab42f68a21d52e1ce9cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:01.255795 8395 tablet_service.cc:1467] Processing CreateTablet for tablet 8f648b1eecab42f68a21d52e1ce9cc89 (DEFAULT_TABLE table=test-table [id=278c82c0abab4fa7908bb0671e94f4b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:01.256807 8395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f648b1eecab42f68a21d52e1ce9cc89. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:01.259284 8517 tablet_bootstrap.cc:492] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: Bootstrap starting.
I20260806 18:45:01.260587 8518 tablet_bootstrap.cc:492] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff: Bootstrap starting.
I20260806 18:45:01.261787 8517 tablet_bootstrap.cc:654] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:01.262573 8517 log.cc:824] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:01.262797 8518 tablet_bootstrap.cc:654] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:01.263581 8518 log.cc:824] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:01.264206 8520 tablet_bootstrap.cc:492] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: Bootstrap starting.
I20260806 18:45:01.264695 8517 tablet_bootstrap.cc:492] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: No bootstrap required, opened a new log
I20260806 18:45:01.264921 8517 ts_tablet_manager.cc:1397] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:45:01.265697 8518 tablet_bootstrap.cc:492] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff: No bootstrap required, opened a new log
I20260806 18:45:01.265884 8518 ts_tablet_manager.cc:1397] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:45:01.266469 8520 tablet_bootstrap.cc:654] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:01.267768 8520 log.cc:824] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:01.268811 8517 raft_consensus.cc:348] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.269049 8517 raft_consensus.cc:374] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:01.269109 8517 raft_consensus.cc:729] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1093b85ebc8a49fca2b1707f38e7f2fd, State: Initialized, Role: FOLLOWER
I20260806 18:45:01.269544 8517 consensus_queue.cc:260] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.269889 8520 tablet_bootstrap.cc:492] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: No bootstrap required, opened a new log
I20260806 18:45:01.270092 8520 ts_tablet_manager.cc:1397] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:45:01.270100 8518 raft_consensus.cc:348] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.270437 8518 raft_consensus.cc:374] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:01.270500 8518 raft_consensus.cc:729] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be357c1e12a24fd49156fa0b09058bff, State: Initialized, Role: FOLLOWER
I20260806 18:45:01.270918 8518 consensus_queue.cc:260] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.271855 8518 ts_tablet_manager.cc:1428] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:45:01.272212 8517 ts_tablet_manager.cc:1428] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:45:01.275097 8520 raft_consensus.cc:348] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.275384 8520 raft_consensus.cc:374] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:01.275452 8520 raft_consensus.cc:729] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12e525cbded747a2a0cd2c0ebd1c7d8d, State: Initialized, Role: FOLLOWER
I20260806 18:45:01.275983 8520 consensus_queue.cc:260] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.276324 8524 raft_consensus.cc:482] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:01.276470 8524 raft_consensus.cc:504] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.276950 8520 ts_tablet_manager.cc:1428] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:45:01.277273 8487 heartbeater.cc:503] Master 127.1.245.126:41133 was elected leader, sending a full tablet report...
I20260806 18:45:01.277611 8524 leader_election.cc:288] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be357c1e12a24fd49156fa0b09058bff (127.1.245.65:46615), 12e525cbded747a2a0cd2c0ebd1c7d8d (127.1.245.67:34829)
I20260806 18:45:01.282457 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.278217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:34809 (local address 127.1.245.65:46615)
0806 18:45:01.278354 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.278359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.278589 (+ 230us) server_negotiation.cc:407] Connection header received
0806 18:45:01.278729 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.278732 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.278786 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.278873 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:01.279177 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.279860 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.281252 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.281893 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.282022 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.282078 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.282246 (+ 168us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.282338 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:45:01.282464 8528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.277846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:46615 (local address 127.1.245.66:34809)
0806 18:45:01.278502 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.278517 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:01.278625 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:01.278896 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:01.278899 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:01.278918 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:01.279053 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.279060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.279981 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.279984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.281120 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.281131 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.282037 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.282040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.282119 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.282143 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:45:01.282219 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:45:01.282274 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"mutex_wait_us":26,"thread_start_us":121,"threads_started":1}
I20260806 18:45:01.283035 8529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.277883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:34829 (local address 127.1.245.66:35227)
0806 18:45:01.278874 (+ 991us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.278889 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:01.278997 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:01.279280 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:01.279283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:01.279302 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:01.279424 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.279430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.280239 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.280243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.281124 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:01.281135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.282617 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.282621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:01.282728 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.282753 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:45:01.282837 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:45:01.282906 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":34,"thread_start_us":126,"threads_started":1}
I20260806 18:45:01.283059 8511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.278296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:35227 (local address 127.1.245.67:34829)
0806 18:45:01.278512 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.278517 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.279082 (+ 565us) server_negotiation.cc:407] Connection header received
0806 18:45:01.279133 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.279136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.279182 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.279271 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:01.279539 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.280131 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.281267 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.281892 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.282022 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.282078 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.282861 (+ 783us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.282942 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260806 18:45:01.283125 8057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f648b1eecab42f68a21d52e1ce9cc89" candidate_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be357c1e12a24fd49156fa0b09058bff" is_pre_election: true
I20260806 18:45:01.283473 8057 raft_consensus.cc:2454] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1093b85ebc8a49fca2b1707f38e7f2fd in term 0.
I20260806 18:45:01.283699 8422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f648b1eecab42f68a21d52e1ce9cc89" candidate_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" is_pre_election: true
I20260806 18:45:01.283967 8422 raft_consensus.cc:2454] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1093b85ebc8a49fca2b1707f38e7f2fd in term 0.
I20260806 18:45:01.283998 8149 leader_election.cc:302] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1093b85ebc8a49fca2b1707f38e7f2fd, be357c1e12a24fd49156fa0b09058bff; no voters:
I20260806 18:45:01.284310 8524 raft_consensus.cc:2781] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:01.284385 8524 raft_consensus.cc:482] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:01.284422 8524 raft_consensus.cc:3037] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:01.285421 8524 raft_consensus.cc:504] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.285714 8524 leader_election.cc:288] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [CANDIDATE]: Term 1 election: Requested vote from peers be357c1e12a24fd49156fa0b09058bff (127.1.245.65:46615), 12e525cbded747a2a0cd2c0ebd1c7d8d (127.1.245.67:34829)
I20260806 18:45:01.286087 8422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f648b1eecab42f68a21d52e1ce9cc89" candidate_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d"
I20260806 18:45:01.286087 8057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f648b1eecab42f68a21d52e1ce9cc89" candidate_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be357c1e12a24fd49156fa0b09058bff"
I20260806 18:45:01.286202 8422 raft_consensus.cc:3037] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:01.286202 8057 raft_consensus.cc:3037] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:01.287506 8422 raft_consensus.cc:2454] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1093b85ebc8a49fca2b1707f38e7f2fd in term 1.
I20260806 18:45:01.287506 8057 raft_consensus.cc:2454] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1093b85ebc8a49fca2b1707f38e7f2fd in term 1.
I20260806 18:45:01.287824 8150 leader_election.cc:302] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1093b85ebc8a49fca2b1707f38e7f2fd, 12e525cbded747a2a0cd2c0ebd1c7d8d; no voters:
I20260806 18:45:01.287967 8524 raft_consensus.cc:2781] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:01.288189 8524 raft_consensus.cc:686] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 1 LEADER]: Becoming Leader. State: Replica: 1093b85ebc8a49fca2b1707f38e7f2fd, State: Running, Role: LEADER
I20260806 18:45:01.288519 8524 consensus_queue.cc:237] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } }
I20260806 18:45:01.291555 7852 catalog_manager.cc:5495] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1093b85ebc8a49fca2b1707f38e7f2fd (127.1.245.66). New cstate: current_term: 1 leader_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1093b85ebc8a49fca2b1707f38e7f2fd" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46389 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 } health_report { overall_health: UNKNOWN } } }
I20260806 18:45:01.305459 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.301970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42610 (local address 127.1.245.66:46389)
0806 18:45:01.302088 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.302093 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.302106 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:01.302255 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.302257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.302374 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.302462 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:01.302723 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.303216 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.304039 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:01.304246 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:01.304340 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:01.304632 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:01.305053 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:01.305172 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.305241 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.305307 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:45:01.311638 8057 raft_consensus.cc:1261] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [term 1 FOLLOWER]: Refusing update from remote peer 1093b85ebc8a49fca2b1707f38e7f2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:01.311693 8422 raft_consensus.cc:1261] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 1 FOLLOWER]: Refusing update from remote peer 1093b85ebc8a49fca2b1707f38e7f2fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:01.312476 8531 consensus_queue.cc:1035] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "be357c1e12a24fd49156fa0b09058bff" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:01.312755 8524 consensus_queue.cc:1035] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "12e525cbded747a2a0cd2c0ebd1c7d8d" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:01.322667 8535 mvcc.cc:205] Tried to move back new op lower bound from 7315627627764117504 to 7315627627680423936. Current Snapshot: MvccSnapshot[applied={T|T < 7315627627764117504}]
I20260806 18:45:01.325019 8536 mvcc.cc:205] Tried to move back new op lower bound from 7315627627764117504 to 7315627627680423936. Current Snapshot: MvccSnapshot[applied={T|T < 7315627627764117504 or (T in {7315627627764117504})}]
I20260806 18:45:01.331020 7852 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:01.331190 7852 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44266:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:01.333702 7852 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a0408320ac064a7391f37701925a965b: Sending DeleteTablet for 3 replicas of tablet 8f648b1eecab42f68a21d52e1ce9cc89
I20260806 18:45:01.334455 8212 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f648b1eecab42f68a21d52e1ce9cc89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:01 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:42604
I20260806 18:45:01.334503 8030 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f648b1eecab42f68a21d52e1ce9cc89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:01 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:44150
I20260806 18:45:01.334836 8395 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f648b1eecab42f68a21d52e1ce9cc89 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:01 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:43194
I20260806 18:45:01.335031 8549 tablet_replica.cc:330] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: stopping tablet replica
I20260806 18:45:01.335291 8556 tablet_replica.cc:330] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: stopping tablet replica
I20260806 18:45:01.335336 8549 raft_consensus.cc:2229] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:01.335666 8556 raft_consensus.cc:2229] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:01.335739 8549 raft_consensus.cc:2258] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:01.336071 8556 raft_consensus.cc:2258] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:01.336079 8551 tablet_replica.cc:330] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff: stopping tablet replica
I20260806 18:45:01.336123 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 7945
I20260806 18:45:01.336347 8551 raft_consensus.cc:2229] T 8f648b1eecab42f68a21d52e1ce9cc89 P be357c1e12a24fd49156fa0b09058bff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:01.345788 8556 ts_tablet_manager.cc:1905] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:01.346741 8549 ts_tablet_manager.cc:1905] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:01.349059 8556 ts_tablet_manager.cc:1918] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:01.349251 8556 log.cc:1196] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/8f648b1eecab42f68a21d52e1ce9cc89
I20260806 18:45:01.349309 8549 ts_tablet_manager.cc:1918] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:01.349421 8549 log.cc:1196] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/8f648b1eecab42f68a21d52e1ce9cc89
I20260806 18:45:01.349730 8549 ts_tablet_manager.cc:1939] T 8f648b1eecab42f68a21d52e1ce9cc89 P 1093b85ebc8a49fca2b1707f38e7f2fd: Deleting consensus metadata
I20260806 18:45:01.349737 8556 ts_tablet_manager.cc:1939] T 8f648b1eecab42f68a21d52e1ce9cc89 P 12e525cbded747a2a0cd2c0ebd1c7d8d: Deleting consensus metadata
I20260806 18:45:01.350980 7835 catalog_manager.cc:4841] TS 1093b85ebc8a49fca2b1707f38e7f2fd (127.1.245.66:46389): tablet 8f648b1eecab42f68a21d52e1ce9cc89 (table test-table [id=278c82c0abab4fa7908bb0671e94f4b5]) successfully deleted
I20260806 18:45:01.351405 7834 catalog_manager.cc:4841] TS 12e525cbded747a2a0cd2c0ebd1c7d8d (127.1.245.67:34829): tablet 8f648b1eecab42f68a21d52e1ce9cc89 (table test-table [id=278c82c0abab4fa7908bb0671e94f4b5]) successfully deleted
W20260806 18:45:01.355329 7833 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:46615 (error 108)
I20260806 18:45:01.356148 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 8130
I20260806 18:45:01.356405 8504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.356102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:46615 (local address 127.0.0.1:44154)
0806 18:45:01.356249 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.356324 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:46615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 18:45:01.361907 7833 catalog_manager.cc:4584] TS be357c1e12a24fd49156fa0b09058bff (127.1.245.65:46615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f648b1eecab42f68a21d52e1ce9cc89: Network error: Client connection negotiation failed: client connection to 127.1.245.65:46615: connect: Connection refused (error 111)
I20260806 18:45:01.367472 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 8303
I20260806 18:45:01.378590 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 7812
2026-08-06T18:45:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (3480 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T18:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:01Z Disabled control of system clock
I20260806 18:45:01.416253 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:45529
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:45529
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:45:01.528288 8563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:01.528577 8563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:01.528614 8563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:01.532750 8563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:01.532809 8563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:01.532823 8563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:01.532840 8563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:01.532855 8563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:01.537669 8563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:45529
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:45529
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:01.538910 8563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:01.540345 8563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:01.549021 8568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:01.549134 8572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:01.549141 8569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:01.549443 8563 server_base.cc:1034] running on GCE node
I20260806 18:45:01.550028 8563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:01.551282 8563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:01.552460 8563 hybrid_clock.cc:648] HybridClock initialized: now 1786041901552443 us; error 28 us; skew 500 ppm
I20260806 18:45:01.554420 8563 webserver.cc:460] Webserver started at http://127.1.245.126:43317/ using document root <none> and password file <none>
I20260806 18:45:01.554982 8563 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:01.555040 8563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:01.555210 8563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:01.557233 8563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "190fb69c0cad1282480bf9b14de56685"
server_key_iv: "b2a2afa15c18a0e5065a7ba1f18bb886"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.557747 8563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "190fb69c0cad1282480bf9b14de56685"
server_key_iv: "b2a2afa15c18a0e5065a7ba1f18bb886"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.561444 8563 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:45:01.564008 8580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.565315 8563 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:01.565482 8563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "190fb69c0cad1282480bf9b14de56685"
server_key_iv: "b2a2afa15c18a0e5065a7ba1f18bb886"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.565593 8563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:01.570207 8563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:01.570848 8563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:01.571012 8563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:01.578060 8563 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:45529
I20260806 18:45:01.578071 8648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:45529 every 8 connection(s)
I20260806 18:45:01.579080 8563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:01.581888 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 8563
I20260806 18:45:01.582006 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:01.582270 2005 external_mini_cluster.cc:1428] Setting key 33259cb6268738a86221d39b67cf4caf
I20260806 18:45:01.582584 8649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:01.588272 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.583769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54238 (local address 127.1.245.126:45529)
0806 18:45:01.584462 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.584478 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.584515 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:45:01.585334 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.585354 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.585703 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.585999 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:01.586881 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:01.586886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:01.586897 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:01.586908 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:01.586965 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:01.586981 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:01.587099 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.587214 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.587580 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":146,"threads_started":1}
I20260806 18:45:01.589460 8649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2: Bootstrap starting.
I20260806 18:45:01.591758 8649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:01.592557 8649 log.cc:824] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:01.594544 8649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2: No bootstrap required, opened a new log
I20260806 18:45:01.598170 8649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45529 } }
I20260806 18:45:01.598439 8649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:01.598498 8649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ebe4a0d7f1940b795c7fcf39ada49e2, State: Initialized, Role: FOLLOWER
I20260806 18:45:01.598909 8649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45529 } }
I20260806 18:45:01.599036 8649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:01.599100 8649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:01.599197 8649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:01.600090 8649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45529 } }
I20260806 18:45:01.600430 8649 leader_election.cc:302] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ebe4a0d7f1940b795c7fcf39ada49e2; no voters:
I20260806 18:45:01.600760 8649 leader_election.cc:288] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:01.600883 8659 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:01.601127 8659 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [term 1 LEADER]: Becoming Leader. State: Replica: 4ebe4a0d7f1940b795c7fcf39ada49e2, State: Running, Role: LEADER
I20260806 18:45:01.601454 8659 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45529 } }
I20260806 18:45:01.602036 8649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:01.603173 8661 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ebe4a0d7f1940b795c7fcf39ada49e2. Latest consensus state: current_term: 1 leader_uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45529 } } }
I20260806 18:45:01.603297 8661 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:01.603272 8660 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ebe4a0d7f1940b795c7fcf39ada49e2" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45529 } } }
I20260806 18:45:01.603368 8660 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:01.603717 8668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:01.607125 8668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:01.613138 8668 catalog_manager.cc:1347] Generated new cluster ID: e246533350d643278b8900c555637a52
I20260806 18:45:01.613224 8668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:01.633339 8668 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:01.633697 8668 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:01.640235 8668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2: Generated new TSK 0
I20260806 18:45:01.641105 8668 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:45:01.646837 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45529
--builtin_ntp_servers=127.1.245.84:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 18:45:01.760870 8681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:01.761152 8681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:01.761207 8681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:01.765367 8681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:01.765443 8681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:01.765522 8681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:01.770515 8681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:01.771672 8681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:01.773145 8681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:01.783484 8688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:01.783586 8687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:01.783471 8691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:01.783836 8681 server_base.cc:1034] running on GCE node
I20260806 18:45:01.784273 8681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:01.784952 8681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:01.786127 8681 hybrid_clock.cc:648] HybridClock initialized: now 1786041901786096 us; error 46 us; skew 500 ppm
I20260806 18:45:01.788190 8681 webserver.cc:460] Webserver started at http://127.1.245.65:45741/ using document root <none> and password file <none>
I20260806 18:45:01.788797 8681 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:01.788865 8681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:01.789063 8681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:01.791241 8681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "098991d0f34de09d3501eeb464bd1d80"
server_key_iv: "21be9ce735428231f5b84b23fc3aba25"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.791817 8681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "098991d0f34de09d3501eeb464bd1d80"
server_key_iv: "21be9ce735428231f5b84b23fc3aba25"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.795734 8681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:45:01.798439 8701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.799837 8681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:45:01.799968 8681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "098991d0f34de09d3501eeb464bd1d80"
server_key_iv: "21be9ce735428231f5b84b23fc3aba25"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.800079 8681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:01.804841 8681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:01.805451 8681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:01.805613 8681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:01.806154 8681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:01.807139 8681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:01.807188 8681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.807241 8681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:01.807269 8681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.816067 8681 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:42071
I20260806 18:45:01.816107 8851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:42071 every 8 connection(s)
I20260806 18:45:01.817005 8681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:01.823490 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.818914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:47931 (local address 127.1.245.126:45529)
0806 18:45:01.819059 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.819063 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.820153 (+ 1090us) server_negotiation.cc:407] Connection header received
0806 18:45:01.821622 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:01.821626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:01.821671 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:01.821748 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:01.822965 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:01.822968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:01.822970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:01.822983 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:01.823005 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:01.823014 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:01.823102 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:01.823311 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 18:45:01.823356 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:45:01.823756 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 8681
I20260806 18:45:01.823864 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:45:01.824141 2005 external_mini_cluster.cc:1428] Setting key 23a3bbfad967cab71f2bc49e4e9737aa
I20260806 18:45:01.824291 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.819338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45529 (local address 127.1.245.65:47931)
0806 18:45:01.820006 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.820045 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:01.821411 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:01.821918 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:01.821924 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:01.822018 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:01.822624 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:01.822627 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:01.822679 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:01.822771 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:01.823110 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:01.823118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:01.823124 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:45:01.823264 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 18:45:01.823460 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":127,"threads_started":1}
I20260806 18:45:01.825568 8852 heartbeater.cc:347] Connected to a master server at 127.1.245.126:45529
I20260806 18:45:01.825785 8852 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:01.826114 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45529
--builtin_ntp_servers=127.1.245.84:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 18:45:01.826165 8852 heartbeater.cc:511] Master 127.1.245.126:45529 requested a full tablet report, sending...
I20260806 18:45:01.827765 8604 ts_manager.cc:194] Registered new tserver with Master: e6c5ef81bcc84a15b6f514ff14bb9f95 (127.1.245.65:42071)
W20260806 18:45:01.939287 8859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:01.939565 8859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:01.939606 8859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:01.943689 8859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:01.943742 8859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:01.943810 8859 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:01.948536 8859 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8859
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:01.949637 8859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:01.951052 8859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:01.960472 8869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:01.960492 8866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:01.960495 8867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:01.960814 8859 server_base.cc:1034] running on GCE node
I20260806 18:45:01.961256 8859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:01.961881 8859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:01.963045 8859 hybrid_clock.cc:648] HybridClock initialized: now 1786041901963016 us; error 42 us; skew 500 ppm
I20260806 18:45:01.968376 8859 webserver.cc:460] Webserver started at http://127.1.245.66:45121/ using document root <none> and password file <none>
I20260806 18:45:01.969007 8859 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:01.969084 8859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:01.969357 8859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:01.971508 8859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "f01d65de78734db298f879226b7e8857"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "073fce2ffa02e39915ef350442f4d7ad"
server_key_iv: "7aba26de1463c7565c5c85b959a6d755"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.972067 8859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "f01d65de78734db298f879226b7e8857"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "073fce2ffa02e39915ef350442f4d7ad"
server_key_iv: "7aba26de1463c7565c5c85b959a6d755"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.975818 8859 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:45:01.978261 8879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.979768 8859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:45:01.979898 8859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "f01d65de78734db298f879226b7e8857"
format_stamp: "Formatted at 2026-08-06 18:45:01 on dist-test-slave-fgb3"
server_key: "073fce2ffa02e39915ef350442f4d7ad"
server_key_iv: "7aba26de1463c7565c5c85b959a6d755"
server_key_version: "encryptionkey@0"
I20260806 18:45:01.979998 8859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:01.984710 8859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:01.985345 8859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:01.985553 8859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:01.986115 8859 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:01.987138 8859 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:01.987187 8859 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.987241 8859 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:01.987267 8859 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:01.995646 8859 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:40417
I20260806 18:45:01.995704 9032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:40417 every 8 connection(s)
I20260806 18:45:01.996582 8859 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:02.002678 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.998437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:43021 (local address 127.1.245.126:45529)
0806 18:45:01.998587 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:01.998592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:01.999757 (+ 1165us) server_negotiation.cc:407] Connection header received
0806 18:45:02.001020 (+ 1263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.001024 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.001062 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.001138 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.002257 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.002261 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.002263 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.002276 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.002297 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.002306 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.002427 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.002524 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.002562 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:45:02.003221 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 8859
I20260806 18:45:02.003333 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:45:02.003604 2005 external_mini_cluster.cc:1428] Setting key 2d15e405d028c9b33fc51f2e68defd87
I20260806 18:45:02.003742 9037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:01.998916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45529 (local address 127.1.245.66:43021)
0806 18:45:01.999612 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:01.999652 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:02.000818 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:02.001307 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:02.001314 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:02.001437 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:02.001985 (+ 548us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:02.001987 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:02.002027 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:02.002094 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:02.002418 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:02.002424 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:02.002428 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:45:02.002572 (+ 144us) client_negotiation.cc:240] Negotiation successful
0806 18:45:02.002796 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260806 18:45:02.004832 9033 heartbeater.cc:347] Connected to a master server at 127.1.245.126:45529
I20260806 18:45:02.005071 9033 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:02.005477 9033 heartbeater.cc:511] Master 127.1.245.126:45529 requested a full tablet report, sending...
I20260806 18:45:02.005494 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45529
--builtin_ntp_servers=127.1.245.84:43079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260806 18:45:02.006582 8604 ts_manager.cc:194] Registered new tserver with Master: f01d65de78734db298f879226b7e8857 (127.1.245.66:40417)
W20260806 18:45:02.117777 9038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:02.118044 9038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:02.118083 9038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:02.122025 9038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:02.122077 9038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:02.122131 9038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:02.126858 9038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:45529
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:02.127934 9038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:02.129292 9038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:02.139170 9043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:02.139145 9045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:02.139148 9049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:02.139446 9038 server_base.cc:1034] running on GCE node
I20260806 18:45:02.140115 9038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:02.140779 9038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:02.141953 9038 hybrid_clock.cc:648] HybridClock initialized: now 1786041902141931 us; error 29 us; skew 500 ppm
I20260806 18:45:02.143855 9038 webserver.cc:460] Webserver started at http://127.1.245.67:40179/ using document root <none> and password file <none>
I20260806 18:45:02.144419 9038 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:02.144480 9038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:02.144670 9038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:02.146803 9038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "6c93ed8f382a46c79d33f475f016b233"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "1f6967e318103cdacb1a8d88762e5387"
server_key_iv: "eb915749a94ffeb07226e0e8209fec5a"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.147370 9038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "6c93ed8f382a46c79d33f475f016b233"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "1f6967e318103cdacb1a8d88762e5387"
server_key_iv: "eb915749a94ffeb07226e0e8209fec5a"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.151125 9038 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:45:02.153697 9059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:02.155114 9038 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:02.155243 9038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "6c93ed8f382a46c79d33f475f016b233"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "1f6967e318103cdacb1a8d88762e5387"
server_key_iv: "eb915749a94ffeb07226e0e8209fec5a"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.155347 9038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:02.159996 9038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:02.160610 9038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:02.160769 9038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:02.161322 9038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:02.162289 9038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:02.162365 9038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:02.162425 9038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:02.162451 9038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:02.171103 9038 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:38631
I20260806 18:45:02.171149 9215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:38631 every 8 connection(s)
I20260806 18:45:02.172029 9038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:02.172571 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 9038
I20260806 18:45:02.172683 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:02.172979 2005 external_mini_cluster.cc:1428] Setting key 35434dc9323a16f0e130a7a25c0479ad
I20260806 18:45:02.178822 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.174506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:46113 (local address 127.1.245.126:45529)
0806 18:45:02.174648 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.174652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.175810 (+ 1158us) server_negotiation.cc:407] Connection header received
0806 18:45:02.177071 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.177073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.177122 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.177200 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.178415 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.178418 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.178421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.178434 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.178455 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.178463 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.178546 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.178667 (+ 121us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.178702 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:45:02.179739 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.174839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:45529 (local address 127.1.245.67:46113)
0806 18:45:02.175671 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.175713 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:02.176872 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:02.177421 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:02.177428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:02.177520 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:02.178069 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:02.178070 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:02.178112 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:02.178185 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:02.178550 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:02.178558 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:02.178562 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:45:02.178701 (+ 139us) client_negotiation.cc:240] Negotiation successful
0806 18:45:02.178914 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":156,"threads_started":1}
I20260806 18:45:02.180873 9219 heartbeater.cc:347] Connected to a master server at 127.1.245.126:45529
I20260806 18:45:02.181087 9219 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:02.181479 9219 heartbeater.cc:511] Master 127.1.245.126:45529 requested a full tablet report, sending...
I20260806 18:45:02.182525 8604 ts_manager.cc:194] Registered new tserver with Master: 6c93ed8f382a46c79d33f475f016b233 (127.1.245.67:38631)
I20260806 18:45:02.187374 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:45:02.190914 8655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.189307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54252 (local address 127.1.245.126:45529)
0806 18:45:02.189441 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.189446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.189602 (+ 156us) server_negotiation.cc:407] Connection header received
0806 18:45:02.189743 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.189748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.189796 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.189871 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.190602 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.190605 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.190607 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.190617 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.190634 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.190641 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.190718 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.190773 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.190805 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:45:02.193809 8604 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54252:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:45:02.196950 8604 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:02.209467 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.205887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:38631 (local address 127.0.0.1:42948)
0806 18:45:02.206480 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.206533 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:02.206642 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:02.207647 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:02.207654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:02.207736 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:02.208322 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:02.208324 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:02.208345 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:02.208421 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:02.209162 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:02.209164 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:02.209167 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:45:02.209238 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:45:02.209303 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":96,"threads_started":1}
I20260806 18:45:02.209511 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.206753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.1.245.67:38631)
0806 18:45:02.207091 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.207106 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.207131 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:45:02.207189 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.207195 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.207484 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.208046 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.208995 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.209001 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.209019 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.209029 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.209064 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.209094 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.209186 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.209253 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.209335 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":120,"threads_started":1}
I20260806 18:45:02.209512 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.206197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.1.245.66:40417)
0806 18:45:02.206679 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.206692 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.207032 (+ 340us) server_negotiation.cc:407] Connection header received
0806 18:45:02.207223 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.207230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.207429 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.207548 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.208894 (+ 1346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.208901 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.208914 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.208927 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.208978 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.209012 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.209129 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.209254 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.209335 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":178,"threads_started":1}
I20260806 18:45:02.210009 9232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.206672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:42071 (local address 127.0.0.1:56818)
0806 18:45:02.207630 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.207644 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:02.207715 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:02.208749 (+ 1034us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:02.208752 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:02.208766 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:02.209368 (+ 602us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:02.209370 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:02.209379 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:02.209402 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:02.209793 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:02.209796 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:02.209799 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:45:02.209861 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:45:02.209893 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":90,"threads_started":1}
I20260806 18:45:02.210212 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.206025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:40417 (local address 127.0.0.1:34414)
0806 18:45:02.206927 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.206941 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:02.207097 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:02.208284 (+ 1187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:02.208287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:02.208302 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:02.208712 (+ 410us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:02.208713 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:02.208724 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:02.208748 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:02.209097 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:02.209107 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:02.209114 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:45:02.209230 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 18:45:02.209313 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"thread_start_us":122,"threads_started":1}
I20260806 18:45:02.210527 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.207080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56818 (local address 127.1.245.65:42071)
0806 18:45:02.207444 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.207456 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.207804 (+ 348us) server_negotiation.cc:407] Connection header received
0806 18:45:02.207864 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.207869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.208050 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.208180 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.209532 (+ 1352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.209538 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.209550 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.209564 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.209611 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.209647 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.209750 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.210249 (+ 499us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.210360 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":155,"threads_started":1}
I20260806 18:45:02.212661 8772 tablet_service.cc:1467] Processing CreateTablet for tablet 172988db21414a9981014627ed39e7de (DEFAULT_TABLE table=test-table [id=03c932c64c624082a5cc94666b044710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:02.212687 9128 tablet_service.cc:1467] Processing CreateTablet for tablet 172988db21414a9981014627ed39e7de (DEFAULT_TABLE table=test-table [id=03c932c64c624082a5cc94666b044710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:02.212852 8943 tablet_service.cc:1467] Processing CreateTablet for tablet 172988db21414a9981014627ed39e7de (DEFAULT_TABLE table=test-table [id=03c932c64c624082a5cc94666b044710]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:02.213761 8943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172988db21414a9981014627ed39e7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:02.213763 8772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172988db21414a9981014627ed39e7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:02.213761 9128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 172988db21414a9981014627ed39e7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:02.220118 9238 tablet_bootstrap.cc:492] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95: Bootstrap starting.
I20260806 18:45:02.221911 9240 tablet_bootstrap.cc:492] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Bootstrap starting.
I20260806 18:45:02.222667 9239 tablet_bootstrap.cc:492] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: Bootstrap starting.
I20260806 18:45:02.222785 9238 tablet_bootstrap.cc:654] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:02.223512 9238 log.cc:824] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:02.224102 9240 tablet_bootstrap.cc:654] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:02.224766 9239 tablet_bootstrap.cc:654] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:02.224938 9240 log.cc:824] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:02.225381 9238 tablet_bootstrap.cc:492] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95: No bootstrap required, opened a new log
I20260806 18:45:02.225589 9238 ts_tablet_manager.cc:1397] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:45:02.225764 9239 log.cc:824] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:02.227360 9240 tablet_bootstrap.cc:492] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: No bootstrap required, opened a new log
I20260806 18:45:02.227548 9240 ts_tablet_manager.cc:1397] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:45:02.227803 9239 tablet_bootstrap.cc:492] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: No bootstrap required, opened a new log
I20260806 18:45:02.227993 9239 ts_tablet_manager.cc:1397] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:45:02.230111 9238 raft_consensus.cc:348] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.230439 9238 raft_consensus.cc:374] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:02.230497 9238 raft_consensus.cc:729] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6c5ef81bcc84a15b6f514ff14bb9f95, State: Initialized, Role: FOLLOWER
I20260806 18:45:02.230948 9240 raft_consensus.cc:348] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.231051 9238 consensus_queue.cc:260] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.231154 9240 raft_consensus.cc:374] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:02.231206 9240 raft_consensus.cc:729] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f01d65de78734db298f879226b7e8857, State: Initialized, Role: FOLLOWER
I20260806 18:45:02.231601 9240 consensus_queue.cc:260] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.232097 9238 ts_tablet_manager.cc:1428] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:45:02.232270 9239 raft_consensus.cc:348] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.232475 8852 heartbeater.cc:503] Master 127.1.245.126:45529 was elected leader, sending a full tablet report...
I20260806 18:45:02.232508 9239 raft_consensus.cc:374] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:02.232551 9239 raft_consensus.cc:729] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c93ed8f382a46c79d33f475f016b233, State: Initialized, Role: FOLLOWER
I20260806 18:45:02.232569 9240 ts_tablet_manager.cc:1428] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260806 18:45:02.232977 9033 heartbeater.cc:503] Master 127.1.245.126:45529 was elected leader, sending a full tablet report...
I20260806 18:45:02.232999 9239 consensus_queue.cc:260] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.233628 9219 heartbeater.cc:503] Master 127.1.245.126:45529 was elected leader, sending a full tablet report...
I20260806 18:45:02.233915 9239 ts_tablet_manager.cc:1428] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260806 18:45:02.247581 9036 tablet.cc:2369] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:45:02.295578 9247 raft_consensus.cc:482] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:02.295846 9247 raft_consensus.cc:504] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.297127 9247 leader_election.cc:288] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f01d65de78734db298f879226b7e8857 (127.1.245.66:40417), e6c5ef81bcc84a15b6f514ff14bb9f95 (127.1.245.65:42071)
I20260806 18:45:02.299233 9237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.297581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:41743 (local address 127.1.245.65:42071)
0806 18:45:02.297760 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.297765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.297784 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:45:02.297841 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.297844 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.297886 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.297990 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.298819 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.298823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.298826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.298841 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.298864 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.298873 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.298959 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.299046 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.299086 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 18:45:02.299224 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.297483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:42071 (local address 127.1.245.67:41743)
0806 18:45:02.297638 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.297657 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:02.297755 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:02.298044 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:02.298047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:02.298061 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:02.298647 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:02.298649 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:02.298662 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:02.298692 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:02.298952 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:02.298956 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:02.298959 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:45:02.299046 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:45:02.299086 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260806 18:45:02.299383 9234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.297593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:57425 (local address 127.1.245.66:40417)
0806 18:45:02.297756 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.297761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.298167 (+ 406us) server_negotiation.cc:407] Connection header received
0806 18:45:02.298296 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.298300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.298368 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.298447 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.299000 (+ 553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.299003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.299005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.299015 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.299033 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.299040 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.299118 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.299182 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.299222 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:45:02.299407 9249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.297484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:40417 (local address 127.1.245.67:57425)
0806 18:45:02.298068 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.298082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:02.298176 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:02.298462 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:02.298465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:02.298477 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:02.298842 (+ 365us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:02.298844 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:02.298853 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:02.298880 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:02.299109 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:02.299112 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:02.299116 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:45:02.299185 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 18:45:02.299219 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":163,"thread_start_us":187,"threads_started":1}
I20260806 18:45:02.299943 8972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172988db21414a9981014627ed39e7de" candidate_uuid: "6c93ed8f382a46c79d33f475f016b233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01d65de78734db298f879226b7e8857" is_pre_election: true
I20260806 18:45:02.299934 8795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172988db21414a9981014627ed39e7de" candidate_uuid: "6c93ed8f382a46c79d33f475f016b233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" is_pre_election: true
I20260806 18:45:02.300226 8972 raft_consensus.cc:2454] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c93ed8f382a46c79d33f475f016b233 in term 0.
I20260806 18:45:02.300232 8795 raft_consensus.cc:2454] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c93ed8f382a46c79d33f475f016b233 in term 0.
I20260806 18:45:02.300701 9065 leader_election.cc:302] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6c93ed8f382a46c79d33f475f016b233, e6c5ef81bcc84a15b6f514ff14bb9f95, f01d65de78734db298f879226b7e8857; no voters:
I20260806 18:45:02.300974 9247 raft_consensus.cc:2781] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:02.301055 9247 raft_consensus.cc:482] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:02.301095 9247 raft_consensus.cc:3037] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:02.302337 9247 raft_consensus.cc:504] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.302785 9247 leader_election.cc:288] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [CANDIDATE]: Term 1 election: Requested vote from peers f01d65de78734db298f879226b7e8857 (127.1.245.66:40417), e6c5ef81bcc84a15b6f514ff14bb9f95 (127.1.245.65:42071)
I20260806 18:45:02.303143 8972 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172988db21414a9981014627ed39e7de" candidate_uuid: "6c93ed8f382a46c79d33f475f016b233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f01d65de78734db298f879226b7e8857"
I20260806 18:45:02.303143 8795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "172988db21414a9981014627ed39e7de" candidate_uuid: "6c93ed8f382a46c79d33f475f016b233" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95"
I20260806 18:45:02.303287 8795 raft_consensus.cc:3037] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:02.303287 8972 raft_consensus.cc:3037] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:02.304677 8795 raft_consensus.cc:2454] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c93ed8f382a46c79d33f475f016b233 in term 1.
I20260806 18:45:02.304677 8972 raft_consensus.cc:2454] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c93ed8f382a46c79d33f475f016b233 in term 1.
I20260806 18:45:02.305015 9065 leader_election.cc:302] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c93ed8f382a46c79d33f475f016b233, f01d65de78734db298f879226b7e8857; no voters:
I20260806 18:45:02.305200 9247 raft_consensus.cc:2781] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:02.305469 9247 raft_consensus.cc:686] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 1 LEADER]: Becoming Leader. State: Replica: 6c93ed8f382a46c79d33f475f016b233, State: Running, Role: LEADER
I20260806 18:45:02.305799 9247 consensus_queue.cc:237] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } }
I20260806 18:45:02.309275 8604 catalog_manager.cc:5495] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c93ed8f382a46c79d33f475f016b233 (127.1.245.67). New cstate: current_term: 1 leader_uuid: "6c93ed8f382a46c79d33f475f016b233" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c93ed8f382a46c79d33f475f016b233" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 } health_report { overall_health: UNKNOWN } } }
W20260806 18:45:02.318222 8853 tablet.cc:2369] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:45:02.338294 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.336834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.1.245.67:38631)
0806 18:45:02.337022 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.337025 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.337039 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:02.337115 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.337117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.337177 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.337255 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.337915 (+ 660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.337918 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.337920 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.337931 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.337949 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.337957 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.338040 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.338104 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.338138 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260806 18:45:02.344535 8795 raft_consensus.cc:1261] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 1 FOLLOWER]: Refusing update from remote peer 6c93ed8f382a46c79d33f475f016b233: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:02.344535 8972 raft_consensus.cc:1261] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 1 FOLLOWER]: Refusing update from remote peer 6c93ed8f382a46c79d33f475f016b233: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:02.345335 9250 consensus_queue.cc:1035] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6c5ef81bcc84a15b6f514ff14bb9f95" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 42071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:02.345532 9247 consensus_queue.cc:1035] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f01d65de78734db298f879226b7e8857" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 40417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:02.353976 9255 mvcc.cc:205] Tried to move back new op lower bound from 7315627631995166720 to 7315627631847833600. Current Snapshot: MvccSnapshot[applied={T|T < 7315627631995166720}]
I20260806 18:45:02.362713 8601 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:02.362931 8601 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54252:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:02.365680 8601 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4ebe4a0d7f1940b795c7fcf39ada49e2: Sending DeleteTablet for 3 replicas of tablet 172988db21414a9981014627ed39e7de
I20260806 18:45:02.366670 9128 tablet_service.cc:1514] Processing DeleteTablet for tablet 172988db21414a9981014627ed39e7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:02 UTC) from {username='slave'} at 127.0.0.1:42948
I20260806 18:45:02.366628 8943 tablet_service.cc:1514] Processing DeleteTablet for tablet 172988db21414a9981014627ed39e7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:02 UTC) from {username='slave'} at 127.0.0.1:34414
I20260806 18:45:02.366709 8772 tablet_service.cc:1514] Processing DeleteTablet for tablet 172988db21414a9981014627ed39e7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:02 UTC) from {username='slave'} at 127.0.0.1:56818
I20260806 18:45:02.367391 9267 tablet_replica.cc:330] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95: stopping tablet replica
I20260806 18:45:02.367705 9267 raft_consensus.cc:2229] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:02.367990 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 8681
I20260806 18:45:02.368062 9267 raft_consensus.cc:2258] T 172988db21414a9981014627ed39e7de P e6c5ef81bcc84a15b6f514ff14bb9f95 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:02.376243 9271 tablet_replica.cc:330] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: stopping tablet replica
I20260806 18:45:02.376699 9271 raft_consensus.cc:2229] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:02.377182 9271 raft_consensus.cc:2258] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:02.378414 9265 tablet_replica.cc:330] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: stopping tablet replica
I20260806 18:45:02.378861 9265 raft_consensus.cc:2229] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:02.379340 9265 raft_consensus.cc:2258] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:02.380878 9265 ts_tablet_manager.cc:1905] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:02.380879 9271 ts_tablet_manager.cc:1905] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:02.388424 9271 ts_tablet_manager.cc:1918] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:02.388423 9265 ts_tablet_manager.cc:1918] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:02.388566 9265 log.cc:1196] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/172988db21414a9981014627ed39e7de
I20260806 18:45:02.388566 9271 log.cc:1196] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/172988db21414a9981014627ed39e7de
I20260806 18:45:02.388906 9271 ts_tablet_manager.cc:1939] T 172988db21414a9981014627ed39e7de P f01d65de78734db298f879226b7e8857: Deleting consensus metadata
I20260806 18:45:02.388916 9265 ts_tablet_manager.cc:1939] T 172988db21414a9981014627ed39e7de P 6c93ed8f382a46c79d33f475f016b233: Deleting consensus metadata
W20260806 18:45:02.389787 8583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:42071 (error 108)
I20260806 18:45:02.390002 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 8859
I20260806 18:45:02.397656 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.396202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:42071 (local address 127.0.0.1:56832)
0806 18:45:02.397463 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.397556 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:42071: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":749}
W20260806 18:45:02.397854 8583 catalog_manager.cc:4584] TS e6c5ef81bcc84a15b6f514ff14bb9f95 (127.1.245.65:42071): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 172988db21414a9981014627ed39e7de: Network error: Client connection negotiation failed: client connection to 127.1.245.65:42071: connect: Connection refused (error 111)
I20260806 18:45:02.407409 8583 catalog_manager.cc:4841] TS 6c93ed8f382a46c79d33f475f016b233 (127.1.245.67:38631): tablet 172988db21414a9981014627ed39e7de (table test-table [id=03c932c64c624082a5cc94666b044710]) successfully deleted
I20260806 18:45:02.415442 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 9038
I20260806 18:45:02.418262 9235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.415753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:40417 (local address 127.0.0.1:34416)
0806 18:45:02.418092 (+ 2339us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.418158 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.66:40417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2224}
I20260806 18:45:02.426508 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 8563
2026-08-06T18:45:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1046 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 18:45:02.454123 2005 mini_oidc.cc:152] Starting JWKS server
I20260806 18:45:02.455917 2005 webserver.cc:460] Webserver started at https://127.0.0.1:39113/ using document root <none> and password file <none>
I20260806 18:45:02.456094 2005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:45:02.456350 2005 webserver.cc:460] Webserver started at http://127.0.0.1:42625/ using document root <none> and password file <none>
2026-08-06T18:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:02Z Disabled control of system clock
I20260806 18:45:02.466106 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:44907
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:44907
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39113/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/testchainca.cert with env {}
W20260806 18:45:02.579818 9283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:02.580098 9283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:02.580137 9283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:02.584270 9283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:02.584344 9283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:02.584359 9283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:02.584375 9283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:02.584389 9283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:45:02.584403 9283 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:45:02.584416 9283 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39113/jwks.json
I20260806 18:45:02.590118 9283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:44907
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:44907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39113/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9283
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:02.591248 9283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:02.592259 9283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:02.600692 9294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:02.600805 9283 server_base.cc:1034] running on GCE node
W20260806 18:45:02.600728 9293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:02.600690 9297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:02.601509 9283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:02.602593 9283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:02.603744 9283 hybrid_clock.cc:648] HybridClock initialized: now 1786041902603731 us; error 25 us; skew 500 ppm
I20260806 18:45:02.605528 9283 webserver.cc:460] Webserver started at http://127.1.245.126:44443/ using document root <none> and password file <none>
I20260806 18:45:02.606074 9283 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:02.606138 9283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:02.606364 9283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:02.608327 9283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "e230c2c488b9437aaec9348612f910b0"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "07eb0f7149e7d4ab826fda6ebaba47e3"
server_key_iv: "7b020b41613e183d6d55b427ac59bf97"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.608844 9283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "e230c2c488b9437aaec9348612f910b0"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "07eb0f7149e7d4ab826fda6ebaba47e3"
server_key_iv: "7b020b41613e183d6d55b427ac59bf97"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.612438 9283 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260806 18:45:02.614698 9305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:02.615875 9283 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:45:02.615994 9283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "e230c2c488b9437aaec9348612f910b0"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "07eb0f7149e7d4ab826fda6ebaba47e3"
server_key_iv: "7b020b41613e183d6d55b427ac59bf97"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.616080 9283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:02.628337 9283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:02.628973 9283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:02.629140 9283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:02.635578 9283 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:44907
I20260806 18:45:02.635648 9370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:44907 every 8 connection(s)
I20260806 18:45:02.636544 9283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:02.640051 9373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:02.642071 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 9283
I20260806 18:45:02.642205 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:02.642565 2005 external_mini_cluster.cc:1428] Setting key 2dc1255b63cdfe81a845f04490906dc9
I20260806 18:45:02.647517 9373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0: Bootstrap starting.
I20260806 18:45:02.650292 9373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:02.651160 9373 log.cc:824] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:02.653049 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.643988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45986 (local address 127.1.245.126:44907)
0806 18:45:02.644673 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.644697 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.644757 (+ 60us) server_negotiation.cc:407] Connection header received
0806 18:45:02.645500 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.645525 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.646040 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.646440 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.646915 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:02.648309 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:02.649231 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:02.649922 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:02.650099 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:02.650967 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.650974 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.650990 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.651490 (+ 500us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.651565 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.651595 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.651692 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.651862 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.652124 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":126,"threads_started":1}
I20260806 18:45:02.653594 9373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0: No bootstrap required, opened a new log
I20260806 18:45:02.655933 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:45:02.657627 9373 raft_consensus.cc:348] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e230c2c488b9437aaec9348612f910b0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44907 } }
I20260806 18:45:02.657850 9373 raft_consensus.cc:374] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:02.657919 9373 raft_consensus.cc:729] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e230c2c488b9437aaec9348612f910b0, State: Initialized, Role: FOLLOWER
I20260806 18:45:02.658375 9373 consensus_queue.cc:260] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e230c2c488b9437aaec9348612f910b0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44907 } }
I20260806 18:45:02.658514 9373 raft_consensus.cc:388] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:02.658582 9373 raft_consensus.cc:482] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:02.658681 9373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:02.659711 9373 raft_consensus.cc:504] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e230c2c488b9437aaec9348612f910b0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44907 } }
I20260806 18:45:02.660048 9373 leader_election.cc:302] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e230c2c488b9437aaec9348612f910b0; no voters:
I20260806 18:45:02.660357 9373 leader_election.cc:288] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:02.660501 9382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:02.660717 9382 raft_consensus.cc:686] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [term 1 LEADER]: Becoming Leader. State: Replica: e230c2c488b9437aaec9348612f910b0, State: Running, Role: LEADER
I20260806 18:45:02.661038 9382 consensus_queue.cc:237] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e230c2c488b9437aaec9348612f910b0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44907 } }
I20260806 18:45:02.661607 9373 sys_catalog.cc:564] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:02.662622 9383 sys_catalog.cc:455] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e230c2c488b9437aaec9348612f910b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e230c2c488b9437aaec9348612f910b0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44907 } } }
I20260806 18:45:02.662776 9383 sys_catalog.cc:458] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:02.663152 9384 sys_catalog.cc:455] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e230c2c488b9437aaec9348612f910b0. Latest consensus state: current_term: 1 leader_uuid: "e230c2c488b9437aaec9348612f910b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e230c2c488b9437aaec9348612f910b0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44907 } } }
I20260806 18:45:02.663828 9384 sys_catalog.cc:458] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:02.663949 9393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:02.666638 9393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:02.671834 9393 catalog_manager.cc:1347] Generated new cluster ID: a150ec2f552b4641a3bccaabaaf86541
I20260806 18:45:02.671908 9393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:02.689971 9393 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:02.691028 9393 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:02.701368 9393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e230c2c488b9437aaec9348612f910b0: Generated new TSK 0
I20260806 18:45:02.702070 9393 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:45:02.733168 9418 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 18:45:02.733343 9418 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:45:02.733914 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.716499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45988 (local address 127.1.245.126:44907)
0806 18:45:02.716628 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.716632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.716768 (+ 136us) server_negotiation.cc:407] Connection header received
0806 18:45:02.716971 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.716975 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.717034 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.717146 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:45:02.717420 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:02.718283 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:02.719144 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:02.719528 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:02.719628 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:02.719979 (+ 351us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:45:02.733472 (+ 13493us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:45:02.733768 (+ 296us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45988: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 18:45:02.734020 9378 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45988: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260806 18:45:02.735157 9416 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:02.716376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:44907 (local address 127.0.0.1:45988)
0806 18:45:02.716692 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:02.716705 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:02.716817 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:02.717169 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:02.717172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:02.717180 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:45:02.717284 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:02.717324 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:02.718432 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:02.718435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:02.719042 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:02.719051 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:02.719636 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:02.719639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:02.719710 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:02.719763 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:45:02.733713 (+ 13950us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 18:45:02.733971 (+ 258us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.245.126:44907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":103,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.245.126:44907: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39113/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 18:45:02.735844 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 9283
2026-08-06T18:45:02Z chronyd exiting
I20260806 18:45:02.781668 2005 test_util.cc:175] -----------------------------------------------
I20260806 18:45:02.781761 2005 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786041874296929-2005-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (329 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 18:45:02.782989 2005 mini_oidc.cc:152] Starting JWKS server
I20260806 18:45:02.783825 2005 webserver.cc:460] Webserver started at https://127.0.0.1:42581/ using document root <none> and password file <none>
I20260806 18:45:02.783924 2005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:45:02.784173 2005 webserver.cc:460] Webserver started at http://127.0.0.1:36777/ using document root <none> and password file <none>
2026-08-06T18:45:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:02Z Disabled control of system clock
I20260806 18:45:02.794173 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:41729
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:33449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:41729
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42581/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/testchainca.cert with env {}
W20260806 18:45:02.906570 9426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:02.906847 9426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:02.906891 9426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:02.910931 9426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:02.910995 9426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:02.911008 9426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:02.911026 9426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:02.911039 9426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:45:02.911054 9426 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:45:02.911067 9426 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42581/jwks.json
I20260806 18:45:02.916457 9426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:41729
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:41729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42581/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:02.917517 9426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:02.918583 9426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:02.927325 9435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:02.927317 9432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:02.927317 9433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:02.927661 9426 server_base.cc:1034] running on GCE node
I20260806 18:45:02.928315 9426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:02.929530 9426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:02.930711 9426 hybrid_clock.cc:648] HybridClock initialized: now 1786041902930683 us; error 39 us; skew 500 ppm
I20260806 18:45:02.932772 9426 webserver.cc:460] Webserver started at http://127.1.245.126:33807/ using document root <none> and password file <none>
I20260806 18:45:02.933373 9426 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:02.933447 9426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:02.933672 9426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:02.935916 9426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "d87b3f8afe1a47ce97451fe2d78bbe03"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "b0ab5ca6005d54380f3036120f4ed950"
server_key_iv: "49546cfc90a1dd9b5ec9a0c10e6c4ad8"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.936559 9426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "d87b3f8afe1a47ce97451fe2d78bbe03"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "b0ab5ca6005d54380f3036120f4ed950"
server_key_iv: "49546cfc90a1dd9b5ec9a0c10e6c4ad8"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.940631 9426 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:45:02.943118 9442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:02.944368 9426 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:02.944490 9426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "d87b3f8afe1a47ce97451fe2d78bbe03"
format_stamp: "Formatted at 2026-08-06 18:45:02 on dist-test-slave-fgb3"
server_key: "b0ab5ca6005d54380f3036120f4ed950"
server_key_iv: "49546cfc90a1dd9b5ec9a0c10e6c4ad8"
server_key_version: "encryptionkey@0"
I20260806 18:45:02.944590 9426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:02.963258 9426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:02.963997 9426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:02.964185 9426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:02.971367 9426 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:41729
I20260806 18:45:02.971418 9521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:41729 every 8 connection(s)
I20260806 18:45:02.972360 9426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:02.975647 9522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:02.980154 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 9426
I20260806 18:45:02.980304 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:02.980582 2005 external_mini_cluster.cc:1428] Setting key 9a81768c2a777e12251a1c382564f37a
I20260806 18:45:02.981904 9522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03: Bootstrap starting.
I20260806 18:45:02.984416 9522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:02.985193 9522 log.cc:824] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:02.987097 9522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03: No bootstrap required, opened a new log
I20260806 18:45:02.989681 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:02.982005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.1.245.126:41729)
0806 18:45:02.982650 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:02.982666 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:02.982702 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:45:02.983366 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:02.983384 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:02.983803 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:02.984099 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:02.984491 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:02.985527 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:02.986427 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:02.986902 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:02.987053 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:02.987948 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:02.987954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:02.987967 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:02.988395 (+ 428us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:02.988456 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:02.988479 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:02.988566 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:02.988706 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 18:45:02.988975 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":103,"threads_started":1}
I20260806 18:45:02.990916 9522 raft_consensus.cc:348] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b3f8afe1a47ce97451fe2d78bbe03" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41729 } }
I20260806 18:45:02.991146 9522 raft_consensus.cc:374] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:02.991202 9522 raft_consensus.cc:729] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d87b3f8afe1a47ce97451fe2d78bbe03, State: Initialized, Role: FOLLOWER
I20260806 18:45:02.991677 9522 consensus_queue.cc:260] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b3f8afe1a47ce97451fe2d78bbe03" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41729 } }
I20260806 18:45:02.991861 9522 raft_consensus.cc:388] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:02.991940 9522 raft_consensus.cc:482] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:02.992039 9522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:02.992533 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:45:02.993098 9522 raft_consensus.cc:504] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b3f8afe1a47ce97451fe2d78bbe03" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41729 } }
I20260806 18:45:02.993448 9522 leader_election.cc:302] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d87b3f8afe1a47ce97451fe2d78bbe03; no voters:
I20260806 18:45:02.993775 9522 leader_election.cc:288] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:02.993875 9528 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:02.994059 9528 raft_consensus.cc:686] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [term 1 LEADER]: Becoming Leader. State: Replica: d87b3f8afe1a47ce97451fe2d78bbe03, State: Running, Role: LEADER
I20260806 18:45:02.994412 9528 consensus_queue.cc:237] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b3f8afe1a47ce97451fe2d78bbe03" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41729 } }
I20260806 18:45:02.995136 9522 sys_catalog.cc:564] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:02.995672 9528 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d87b3f8afe1a47ce97451fe2d78bbe03. Latest consensus state: current_term: 1 leader_uuid: "d87b3f8afe1a47ce97451fe2d78bbe03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b3f8afe1a47ce97451fe2d78bbe03" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41729 } } }
I20260806 18:45:02.995783 9528 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:02.996073 9529 sys_catalog.cc:455] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d87b3f8afe1a47ce97451fe2d78bbe03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d87b3f8afe1a47ce97451fe2d78bbe03" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41729 } } }
I20260806 18:45:02.996189 9529 sys_catalog.cc:458] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:02.996218 9534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:02.999665 9534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:03.005501 9534 catalog_manager.cc:1347] Generated new cluster ID: fe1b63c397554155b893f36a35d24760
I20260806 18:45:03.005575 9534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:03.010129 9534 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:03.011386 9534 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:03.018977 9534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d87b3f8afe1a47ce97451fe2d78bbe03: Generated new TSK 0
I20260806 18:45:03.019855 9534 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 18:45:03.048154 9568 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 18:45:03.048346 9568 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 18:45:03.049010 9526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:03.032791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56044 (local address 127.1.245.126:41729)
0806 18:45:03.032911 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:03.032915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:03.033032 (+ 117us) server_negotiation.cc:407] Connection header received
0806 18:45:03.033186 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:03.033190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:03.033251 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:03.033377 (+ 126us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:45:03.033564 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.034454 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.035320 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.035669 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.035771 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:03.036024 (+ 253us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:45:03.048530 (+ 12506us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:45:03.048845 (+ 315us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56044: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":39}
W20260806 18:45:03.049083 9526 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56044: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260806 18:45:03.049126 9567 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:03.032667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41729 (local address 127.0.0.1:56044)
0806 18:45:03.032940 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:03.032952 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:03.033052 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:03.033371 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:03.033373 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:03.033379 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:45:03.033470 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:03.033474 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.034578 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.034581 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:03.035215 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:03.035225 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.035773 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.035776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:03.035836 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:03.035860 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:45:03.048776 (+ 12916us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 18:45:03.048947 (+ 171us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.245.126:41729: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42581/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
I20260806 18:45:03.049793 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 9426
2026-08-06T18:45:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (314 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 18:45:03.097200 2005 mini_oidc.cc:152] Starting JWKS server
I20260806 18:45:03.098078 2005 webserver.cc:460] Webserver started at https://127.0.0.1:33271/ using document root <none> and password file <none>
I20260806 18:45:03.098165 2005 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 18:45:03.098418 2005 webserver.cc:460] Webserver started at http://127.0.0.1:34103/ using document root <none> and password file <none>
2026-08-06T18:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:03Z Disabled control of system clock
I20260806 18:45:03.107978 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:36081
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:40787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:36081
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33271/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:45:03.223837 9579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:03.224121 9579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:03.224162 9579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:03.228193 9579 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:03.228255 9579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:03.228271 9579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:03.228286 9579 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:03.228300 9579 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:45:03.228315 9579 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:45:03.228327 9579 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33271/jwks.json
I20260806 18:45:03.233152 9579 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:36081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:36081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33271/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:03.234247 9579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:03.235670 9579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:03.244164 9588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:03.244225 9586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:03.244236 9590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:03.244443 9579 server_base.cc:1034] running on GCE node
I20260806 18:45:03.245095 9579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:03.246172 9579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:03.247334 9579 hybrid_clock.cc:648] HybridClock initialized: now 1786041903247312 us; error 35 us; skew 500 ppm
I20260806 18:45:03.249195 9579 webserver.cc:460] Webserver started at http://127.1.245.126:33409/ using document root <none> and password file <none>
I20260806 18:45:03.249761 9579 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:03.249831 9579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:03.250031 9579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:03.252161 9579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "c35c021cf5b24299bf08f2e4355b0ae3"
format_stamp: "Formatted at 2026-08-06 18:45:03 on dist-test-slave-fgb3"
server_key: "24a8a95f03d1f8f27269780350fa68c4"
server_key_iv: "6de06e58b545a00ab6219382f3644949"
server_key_version: "encryptionkey@0"
I20260806 18:45:03.252694 9579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "c35c021cf5b24299bf08f2e4355b0ae3"
format_stamp: "Formatted at 2026-08-06 18:45:03 on dist-test-slave-fgb3"
server_key: "24a8a95f03d1f8f27269780350fa68c4"
server_key_iv: "6de06e58b545a00ab6219382f3644949"
server_key_version: "encryptionkey@0"
I20260806 18:45:03.256589 9579 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:45:03.259157 9598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:03.260377 9579 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:03.260560 9579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "c35c021cf5b24299bf08f2e4355b0ae3"
format_stamp: "Formatted at 2026-08-06 18:45:03 on dist-test-slave-fgb3"
server_key: "24a8a95f03d1f8f27269780350fa68c4"
server_key_iv: "6de06e58b545a00ab6219382f3644949"
server_key_version: "encryptionkey@0"
I20260806 18:45:03.260676 9579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:03.284405 9579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:03.285180 9579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:03.285373 9579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:03.292424 9579 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:36081
I20260806 18:45:03.292438 9668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:36081 every 8 connection(s)
I20260806 18:45:03.293404 9579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:03.293850 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 9579
I20260806 18:45:03.293972 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:03.294258 2005 external_mini_cluster.cc:1428] Setting key 0e82837529fbd2d8584352297ad042ee
I20260806 18:45:03.296998 9669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:03.303144 9669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3: Bootstrap starting.
I20260806 18:45:03.303548 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:03.295692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.1.245.126:36081)
0806 18:45:03.296390 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:03.296406 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:03.296443 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:45:03.297084 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:03.297103 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:03.297523 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:03.297806 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:03.298216 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.299289 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.300173 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.300646 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.300798 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:03.301752 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:03.301757 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:03.301771 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:03.302160 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:03.302220 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:03.302242 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:03.302397 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:03.302536 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 18:45:03.302801 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":163,"threads_started":1}
I20260806 18:45:03.305701 9669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:03.306581 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:45:03.306632 9669 log.cc:824] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:03.308996 9669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3: No bootstrap required, opened a new log
I20260806 18:45:03.312628 9669 raft_consensus.cc:348] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35c021cf5b24299bf08f2e4355b0ae3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 36081 } }
I20260806 18:45:03.312856 9669 raft_consensus.cc:374] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:03.312904 9669 raft_consensus.cc:729] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c35c021cf5b24299bf08f2e4355b0ae3, State: Initialized, Role: FOLLOWER
I20260806 18:45:03.313333 9669 consensus_queue.cc:260] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35c021cf5b24299bf08f2e4355b0ae3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 36081 } }
I20260806 18:45:03.313462 9669 raft_consensus.cc:388] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:03.313524 9669 raft_consensus.cc:482] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:03.313627 9669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:03.314826 9669 raft_consensus.cc:504] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35c021cf5b24299bf08f2e4355b0ae3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 36081 } }
I20260806 18:45:03.315152 9669 leader_election.cc:302] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c35c021cf5b24299bf08f2e4355b0ae3; no voters:
I20260806 18:45:03.315562 9669 leader_election.cc:288] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:03.315721 9681 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:03.315961 9681 raft_consensus.cc:686] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [term 1 LEADER]: Becoming Leader. State: Replica: c35c021cf5b24299bf08f2e4355b0ae3, State: Running, Role: LEADER
I20260806 18:45:03.316327 9681 consensus_queue.cc:237] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35c021cf5b24299bf08f2e4355b0ae3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 36081 } }
I20260806 18:45:03.316879 9669 sys_catalog.cc:564] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:03.318133 9682 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c35c021cf5b24299bf08f2e4355b0ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35c021cf5b24299bf08f2e4355b0ae3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 36081 } } }
I20260806 18:45:03.318269 9682 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:03.318539 9682 sys_catalog.cc:455] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c35c021cf5b24299bf08f2e4355b0ae3. Latest consensus state: current_term: 1 leader_uuid: "c35c021cf5b24299bf08f2e4355b0ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c35c021cf5b24299bf08f2e4355b0ae3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 36081 } } }
I20260806 18:45:03.318632 9682 sys_catalog.cc:458] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3 [sys.catalog]: This master's current role is: LEADER
W20260806 18:45:03.322877 9703 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:45:03.322952 9703 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:45:03.323071 9693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:03.323982 9693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:03.329555 9693 catalog_manager.cc:1347] Generated new cluster ID: b72320611fe946b2ba04f1bd5e794c97
I20260806 18:45:03.329627 9693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:03.338994 9693 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:03.339932 9693 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:03.352777 9693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c35c021cf5b24299bf08f2e4355b0ae3: Generated new TSK 0
I20260806 18:45:03.353633 9693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:45:03.373397 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:03.368060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.1.245.126:36081)
0806 18:45:03.368211 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:03.368216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:03.368350 (+ 134us) server_negotiation.cc:407] Connection header received
0806 18:45:03.368514 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:03.368518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:03.368577 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:03.368696 (+ 119us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:45:03.368879 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.369690 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.370672 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.371014 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.371114 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:03.371450 (+ 336us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:45:03.372997 (+ 1547us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33271/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:45:03.373242 (+ 245us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47406: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33271/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260806 18:45:03.373474 9677 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47406: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33271/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 18:45:03.375218 9714 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:03.367922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:36081 (local address 127.0.0.1:47406)
0806 18:45:03.368267 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:03.368281 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:03.368385 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:03.368686 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:03.368688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:03.368693 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:45:03.368789 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:03.368793 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.369829 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.369832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:03.370545 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:03.370555 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.371152 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.371155 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:03.371233 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:03.371259 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:45:03.373188 (+ 1929us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33271/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 18:45:03.375032 (+ 1844us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.245.126:36081: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33271/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":1616,"thread_start_us":138,"threads_started":1}
I20260806 18:45:03.375964 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 9579
E20260806 18:45:03.386610 9715 webserver.cc:533] Webserver: sslize failed:
E20260806 18:45:03.386699 9715 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T18:45:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (324 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T18:45:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:03Z Disabled control of system clock
I20260806 18:45:03.431209 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:39159
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:39159
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/testchainca.cert with env {}
W20260806 18:45:03.545094 9720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:03.545387 9720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:03.545450 9720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:03.549579 9720 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:03.549651 9720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:03.549677 9720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:03.549703 9720 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:03.549732 9720 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 18:45:03.549767 9720 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 18:45:03.549791 9720 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 18:45:03.555145 9720 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:39159
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:39159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:03.556207 9720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:03.557247 9720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:03.565800 9727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:03.565917 9728 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:03.565800 9733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:03.566089 9720 server_base.cc:1034] running on GCE node
I20260806 18:45:03.566802 9720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:03.567957 9720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:03.569136 9720 hybrid_clock.cc:648] HybridClock initialized: now 1786041903569117 us; error 30 us; skew 500 ppm
I20260806 18:45:03.571174 9720 webserver.cc:460] Webserver started at http://127.1.245.126:37347/ using document root <none> and password file <none>
I20260806 18:45:03.571758 9720 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:03.571830 9720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:03.572103 9720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:03.574096 9720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "43573ea5549d4180aedd1f193cb5e4ae"
format_stamp: "Formatted at 2026-08-06 18:45:03 on dist-test-slave-fgb3"
server_key: "85eaa5907d959daf87fedd6dd6b5280a"
server_key_iv: "e345210e6040658dadaf36470aa7128d"
server_key_version: "encryptionkey@0"
I20260806 18:45:03.574772 9720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "43573ea5549d4180aedd1f193cb5e4ae"
format_stamp: "Formatted at 2026-08-06 18:45:03 on dist-test-slave-fgb3"
server_key: "85eaa5907d959daf87fedd6dd6b5280a"
server_key_iv: "e345210e6040658dadaf36470aa7128d"
server_key_version: "encryptionkey@0"
I20260806 18:45:03.578500 9720 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:45:03.580899 9742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:03.582077 9720 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:03.582202 9720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "43573ea5549d4180aedd1f193cb5e4ae"
format_stamp: "Formatted at 2026-08-06 18:45:03 on dist-test-slave-fgb3"
server_key: "85eaa5907d959daf87fedd6dd6b5280a"
server_key_iv: "e345210e6040658dadaf36470aa7128d"
server_key_version: "encryptionkey@0"
I20260806 18:45:03.582362 9720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:03.598820 9720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:03.599526 9720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:03.599700 9720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:03.606663 9720 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:39159
I20260806 18:45:03.606745 9812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:39159 every 8 connection(s)
I20260806 18:45:03.607625 9720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:03.610800 9813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:03.616365 9813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae: Bootstrap starting.
I20260806 18:45:03.617596 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 9720
I20260806 18:45:03.617715 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:03.618033 2005 external_mini_cluster.cc:1428] Setting key afc08fba57bfb785add4f747fc9f0220
I20260806 18:45:03.619450 9813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:03.620204 9813 log.cc:824] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:03.622407 9813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae: No bootstrap required, opened a new log
I20260806 18:45:03.625834 9813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43573ea5549d4180aedd1f193cb5e4ae" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39159 } }
I20260806 18:45:03.626072 9813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:03.626138 9813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43573ea5549d4180aedd1f193cb5e4ae, State: Initialized, Role: FOLLOWER
I20260806 18:45:03.626583 9813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43573ea5549d4180aedd1f193cb5e4ae" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39159 } }
I20260806 18:45:03.626729 9813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:03.626796 9813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:03.626896 9813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:03.627202 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:03.619125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.1.245.126:39159)
0806 18:45:03.619878 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:03.619893 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:03.619931 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:45:03.620647 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:03.620673 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:03.621149 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:03.621474 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:03.621906 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.622959 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.623872 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.624349 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.624505 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:03.625292 (+ 787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:03.625297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:03.625311 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:03.625701 (+ 390us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:03.625759 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:03.625792 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:03.625877 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:03.626020 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:45:03.626307 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":153,"threads_started":1}
I20260806 18:45:03.628031 9813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43573ea5549d4180aedd1f193cb5e4ae" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39159 } }
I20260806 18:45:03.628373 9813 leader_election.cc:302] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43573ea5549d4180aedd1f193cb5e4ae; no voters:
I20260806 18:45:03.628669 9813 leader_election.cc:288] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:03.628803 9820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:03.629011 9820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [term 1 LEADER]: Becoming Leader. State: Replica: 43573ea5549d4180aedd1f193cb5e4ae, State: Running, Role: LEADER
I20260806 18:45:03.629307 9820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43573ea5549d4180aedd1f193cb5e4ae" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39159 } }
I20260806 18:45:03.629944 9813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:03.630271 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:45:03.630601 9820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43573ea5549d4180aedd1f193cb5e4ae. Latest consensus state: current_term: 1 leader_uuid: "43573ea5549d4180aedd1f193cb5e4ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43573ea5549d4180aedd1f193cb5e4ae" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39159 } } }
I20260806 18:45:03.630716 9820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:03.631101 9821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43573ea5549d4180aedd1f193cb5e4ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43573ea5549d4180aedd1f193cb5e4ae" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39159 } } }
I20260806 18:45:03.631263 9821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:03.631108 9829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:03.634289 9829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:03.640203 9829 catalog_manager.cc:1347] Generated new cluster ID: 9fb18e4f780b43bebfaee435f0efbc1c
I20260806 18:45:03.640266 9829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:03.654302 9829 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:03.655260 9829 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:03.661710 9829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43573ea5549d4180aedd1f193cb5e4ae: Generated new TSK 0
I20260806 18:45:03.662396 9829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:45:03.821986 9816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:03.672811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56596 (local address 127.1.245.126:39159)
0806 18:45:03.672949 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:03.672954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:03.673116 (+ 162us) server_negotiation.cc:407] Connection header received
0806 18:45:03.673274 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:03.673276 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:03.673322 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:03.673408 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0806 18:45:03.673638 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.674190 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.675419 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.675633 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.675706 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:03.676006 (+ 300us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 18:45:03.821561 (+145555us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:45:03.821859 (+ 298us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56596: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:45:03.822077 9816 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56596: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 18:45:03.822238 9855 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:03.672723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:39159 (local address 127.0.0.1:56596)
0806 18:45:03.673032 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:03.673047 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:03.673162 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:03.673425 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:03.673428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:03.673435 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0806 18:45:03.673549 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:03.673555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.674304 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.674308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:03.675315 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:03.675326 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:03.675734 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:03.675738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:03.675832 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:03.675861 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 18:45:03.821852 (+145991us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 18:45:03.822057 (+ 205us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.245.126:39159: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":94,"threads_started":1}
I20260806 18:45:03.823015 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 9720
2026-08-06T18:45:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (429 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (128 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:45:04 dist-test-slave-fgb3 krb5kdc[9875](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:45:04 dist-test-slave-fgb3 krb5kdc[9875](info): set up 2 sockets
Aug 06 18:45:04 dist-test-slave-fgb3 krb5kdc[9875](info): commencing operation
krb5kdc: starting...
W20260806 18:45:05.208084 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:45:05 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:45:05.338486 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:37305
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:37305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:05.453783 9893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:05.454082 9893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:05.454138 9893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:05.458235 9893 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:05.458343 9893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:05.458381 9893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:05.458413 9893 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:05.458442 9893 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:05.463887 9893 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:37305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:37305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9893
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:05.464986 9893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:05.466068 9893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:05.474727 9898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:05.474732 9899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:05.474871 9893 server_base.cc:1034] running on GCE node
W20260806 18:45:05.474727 9901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:05.475617 9893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:05.476805 9893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:05.478018 9893 hybrid_clock.cc:648] HybridClock initialized: now 1786041905478004 us; error 41 us; skew 500 ppm
Aug 06 18:45:05 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:05.480996 9893 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:45:05.482026 9893 webserver.cc:460] Webserver started at http://127.1.245.126:46693/ using document root <none> and password file <none>
I20260806 18:45:05.482656 9893 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:05.482728 9893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:05.482952 9893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:05.485154 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "1d8ca44e74f848a499a91af306904e57"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "943ef0d1aa582f443a58f36c9a8c1a16"
server_key_iv: "5fced2bfded759a0ecb505e84fd781c0"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.485737 9893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "1d8ca44e74f848a499a91af306904e57"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "943ef0d1aa582f443a58f36c9a8c1a16"
server_key_iv: "5fced2bfded759a0ecb505e84fd781c0"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.489722 9893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260806 18:45:05.492399 9908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:05.493640 9893 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:05.493764 9893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "1d8ca44e74f848a499a91af306904e57"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "943ef0d1aa582f443a58f36c9a8c1a16"
server_key_iv: "5fced2bfded759a0ecb505e84fd781c0"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.493863 9893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:05.505157 9893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:05.508289 9893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:05.508487 9893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:05.515714 9893 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:37305
I20260806 18:45:05.515727 9984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:37305 every 8 connection(s)
I20260806 18:45:05.516675 9893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:05.519862 9989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:05.525655 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 9893
I20260806 18:45:05.525713 9989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57: Bootstrap starting.
I20260806 18:45:05.525791 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:05.526090 2005 external_mini_cluster.cc:1428] Setting key be14dafb8072056e1072d946b0a6303c
I20260806 18:45:05.528544 9989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:05.529287 9989 log.cc:824] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:05.531216 9989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57: No bootstrap required, opened a new log
Aug 06 18:45:05 dist-test-slave-fgb3 krb5kdc[9875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:05.534984 9989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d8ca44e74f848a499a91af306904e57" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37305 } }
I20260806 18:45:05.535234 9989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:05.535291 9989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d8ca44e74f848a499a91af306904e57, State: Initialized, Role: FOLLOWER
I20260806 18:45:05.535813 9989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d8ca44e74f848a499a91af306904e57" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37305 } }
I20260806 18:45:05.535970 9989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:05.536037 9989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:05.536139 9989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:05.537017 9989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d8ca44e74f848a499a91af306904e57" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37305 } }
I20260806 18:45:05.537365 9989 leader_election.cc:302] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1d8ca44e74f848a499a91af306904e57; no voters:
I20260806 18:45:05.537676 9989 leader_election.cc:288] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:05.537811 9995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:05.538013 9995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [term 1 LEADER]: Becoming Leader. State: Replica: 1d8ca44e74f848a499a91af306904e57, State: Running, Role: LEADER
I20260806 18:45:05.538359 9995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d8ca44e74f848a499a91af306904e57" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37305 } }
I20260806 18:45:05.539090 9989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:05.540046 9997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1d8ca44e74f848a499a91af306904e57. Latest consensus state: current_term: 1 leader_uuid: "1d8ca44e74f848a499a91af306904e57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d8ca44e74f848a499a91af306904e57" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37305 } } }
I20260806 18:45:05.540403 9997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:05.540611 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:05.527605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47522 (local address 127.1.245.126:37305)
0806 18:45:05.528092 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:05.528114 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:05.528153 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:45:05.528850 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:05.528870 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:05.529227 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:05.529568 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:05.530402 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:05.531395 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:05.532237 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:05.532701 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:05.532850 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:05.535473 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:05.535480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:05.535496 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:05.535531 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:05.537285 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:05.537741 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:05.537747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:05.537754 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:05.537912 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:05.538282 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:05.538285 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:05.538287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:05.538779 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:05.538998 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:05.539239 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 18:45:05.539563 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":161,"threads_started":1}
I20260806 18:45:05.540310 9996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1d8ca44e74f848a499a91af306904e57" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1d8ca44e74f848a499a91af306904e57" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37305 } } }
I20260806 18:45:05.540975 9996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:05.541793 10006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:05.544433 10006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:05.550137 10006 catalog_manager.cc:1347] Generated new cluster ID: a44d2c40c22040f089c24f31646caa3b
I20260806 18:45:05.550218 10006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:05.560842 10006 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:05.561829 10006 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:05.567176 10006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1d8ca44e74f848a499a91af306904e57: Generated new TSK 0
I20260806 18:45:05.567950 10006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:45:05.607466 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37305
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:05.747179 10023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:05.747442 10023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:05.747485 10023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:05.751439 10023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:05.751499 10023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:05.751559 10023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:05.756729 10023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:05.757786 10023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:05.758854 10023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:45:05.768587 10023 server_base.cc:1034] running on GCE node
W20260806 18:45:05.768548 10030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:05.768653 10036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:05.768550 10032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:05.769238 10023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:05.769855 10023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:05.771008 10023 hybrid_clock.cc:648] HybridClock initialized: now 1786041905770989 us; error 27 us; skew 500 ppm
Aug 06 18:45:05 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:05.773712 10023 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:45:05.774641 10023 webserver.cc:460] Webserver started at http://127.1.245.65:41589/ using document root <none> and password file <none>
I20260806 18:45:05.775197 10023 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:05.775265 10023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:05.775481 10023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:05.777551 10023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "083e6c2a574e453da0a6341afe025cfd"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "4741c11f2eefeba57a7218a41e9e943d"
server_key_iv: "6b2da40459606e7314055d9f18d81f2f"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.778113 10023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "083e6c2a574e453da0a6341afe025cfd"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "4741c11f2eefeba57a7218a41e9e943d"
server_key_iv: "6b2da40459606e7314055d9f18d81f2f"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.781848 10023 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:45:05.784334 10046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:05.785602 10023 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:45:05.785713 10023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "083e6c2a574e453da0a6341afe025cfd"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "4741c11f2eefeba57a7218a41e9e943d"
server_key_iv: "6b2da40459606e7314055d9f18d81f2f"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.785810 10023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:05.795225 10023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:05.798398 10023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:05.798579 10023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:05.799157 10023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:05.800020 10023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:05.800071 10023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:05.800125 10023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:05.800153 10023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:05.808593 10023 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:34751
I20260806 18:45:05.808619 10197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:34751 every 8 connection(s)
I20260806 18:45:05.809438 10023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:05.815296 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 10023
I20260806 18:45:05.815416 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:45:05.815676 2005 external_mini_cluster.cc:1428] Setting key 6d6beb3504c5c18f5058328e34b4be17
Aug 06 18:45:05 dist-test-slave-fgb3 krb5kdc[9875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:05.824005 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:05.811195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:60209 (local address 127.1.245.126:37305)
0806 18:45:05.811362 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:05.811367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:05.812423 (+ 1056us) server_negotiation.cc:407] Connection header received
0806 18:45:05.813907 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:05.813910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:05.813960 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:05.814036 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:05.815223 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:05.815819 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:05.817307 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:05.817534 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:05.817629 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:05.820353 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:05.820357 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:05.820359 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:05.820386 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:05.821832 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:05.822358 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:05.822362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:05.822364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:05.822457 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:05.822794 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:05.822797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:05.822799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:05.823050 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:05.823156 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:05.823685 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 18:45:05.823826 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:45:05.824771 10202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:05.811668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37305 (local address 127.1.245.65:60209)
0806 18:45:05.812201 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:05.812260 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:05.813671 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:05.814181 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:05.814188 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:05.814582 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:05.815073 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:05.815087 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:05.815941 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:05.815944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:05.817192 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:05.817202 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:05.817681 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:05.817686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:05.817866 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:05.818501 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:05.818518 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:05.820186 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:05.821956 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:05.821962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:05.821978 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:05.822246 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:05.822553 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:05.822556 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:05.822557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:05.822701 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:05.823163 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:05.823168 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:05.823449 (+ 281us) client_negotiation.cc:769] Sending connection context
0806 18:45:05.823613 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 18:45:05.823880 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:45:05.825984 10198 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37305
I20260806 18:45:05.826224 10198 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:05.826756 10198 heartbeater.cc:511] Master 127.1.245.126:37305 requested a full tablet report, sending...
I20260806 18:45:05.828380 9934 ts_manager.cc:194] Registered new tserver with Master: 083e6c2a574e453da0a6341afe025cfd (127.1.245.65:34751)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:45:05.829762 9934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:60209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:45:05.846230 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37305
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:05.958734 10209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:05.958989 10209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:05.959031 10209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:05.962967 10209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:05.963021 10209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:05.963088 10209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:05.968184 10209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:05.969197 10209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:05.970237 10209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:05.979545 10216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:05.979553 10220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:05.979578 10218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:05.979805 10209 server_base.cc:1034] running on GCE node
I20260806 18:45:05.980294 10209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:05.980907 10209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:05.982084 10209 hybrid_clock.cc:648] HybridClock initialized: now 1786041905982059 us; error 42 us; skew 500 ppm
Aug 06 18:45:05 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:05.984925 10209 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:45:05.985904 10209 webserver.cc:460] Webserver started at http://127.1.245.66:41709/ using document root <none> and password file <none>
I20260806 18:45:05.986572 10209 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:05.986670 10209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:05.986904 10209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:05.989018 10209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "9119efe2bfcc486e81623a8e0f61e020"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "548d721c1b45dec823eca5d54d4a9c85"
server_key_iv: "ca8329118e7a590eca5194eb9e2c7884"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.989621 10209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "9119efe2bfcc486e81623a8e0f61e020"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "548d721c1b45dec823eca5d54d4a9c85"
server_key_iv: "ca8329118e7a590eca5194eb9e2c7884"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.993618 10209 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:45:05.996277 10229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:05.997509 10209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:45:05.997614 10209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "9119efe2bfcc486e81623a8e0f61e020"
format_stamp: "Formatted at 2026-08-06 18:45:05 on dist-test-slave-fgb3"
server_key: "548d721c1b45dec823eca5d54d4a9c85"
server_key_iv: "ca8329118e7a590eca5194eb9e2c7884"
server_key_version: "encryptionkey@0"
I20260806 18:45:05.997716 10209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:06.021901 10209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:06.025353 10209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:06.025549 10209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:06.026223 10209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:06.027140 10209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:06.027191 10209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:06.027231 10209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:06.027246 10209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:06.036084 10209 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:35713
I20260806 18:45:06.036128 10395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:35713 every 8 connection(s)
I20260806 18:45:06.037017 10209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:06.042968 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 10209
I20260806 18:45:06.043084 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:45:06.043380 2005 external_mini_cluster.cc:1428] Setting key 7ea75836316ff4e209c68fff6760b6af
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041905, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:06.052386 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:06.039060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:48575 (local address 127.1.245.126:37305)
0806 18:45:06.039210 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:06.039214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:06.040329 (+ 1115us) server_negotiation.cc:407] Connection header received
0806 18:45:06.041826 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:06.041830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:06.041881 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:06.041946 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:06.043530 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.044158 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.045645 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.045864 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.045943 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:06.048564 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:06.048568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:06.048570 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:06.048597 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:06.050098 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.050714 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.050717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.050718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.050798 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.051165 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.051167 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.051169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.051376 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:06.051450 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:06.052108 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 18:45:06.052225 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:45:06.053551 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:06.039522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37305 (local address 127.1.245.66:48575)
0806 18:45:06.040133 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:06.040195 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:06.041593 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:06.042114 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:06.042121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:06.042662 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:06.043371 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:06.043394 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.044281 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.044284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:06.045526 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:06.045538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.046005 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.046008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:06.046193 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:06.046882 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:06.046899 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:06.048375 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:06.050222 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:06.050228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:06.050246 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:06.050620 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:06.050883 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:06.050886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:06.050888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:06.051080 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:06.051474 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:06.051481 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:06.051829 (+ 348us) client_negotiation.cc:769] Sending connection context
0806 18:45:06.052042 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 18:45:06.052398 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":151,"threads_started":1}
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:45:06.054819 10396 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37305
I20260806 18:45:06.055074 10396 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:06.055631 10396 heartbeater.cc:511] Master 127.1.245.126:37305 requested a full tablet report, sending...
I20260806 18:45:06.056789 9934 ts_manager.cc:194] Registered new tserver with Master: 9119efe2bfcc486e81623a8e0f61e020 (127.1.245.66:35713)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
I20260806 18:45:06.057499 9934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:48575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:45:06.074162 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37305
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:06.188002 10403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:06.188323 10403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:06.188371 10403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:06.192878 10403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:06.192967 10403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:06.193034 10403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:06.198822 10403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37305
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:06.199985 10403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:06.201085 10403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:06.210731 10409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:06.210721 10413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:06.210721 10410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:06.211198 10403 server_base.cc:1034] running on GCE node
I20260806 18:45:06.211669 10403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:06.212349 10403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:06.213527 10403 hybrid_clock.cc:648] HybridClock initialized: now 1786041906213497 us; error 42 us; skew 500 ppm
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:06.216459 10403 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:45:06.217411 10403 webserver.cc:460] Webserver started at http://127.1.245.67:37105/ using document root <none> and password file <none>
I20260806 18:45:06.217991 10403 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:06.218061 10403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:06.218286 10403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:06.220388 10403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "a9e0aaa7eb4346b89125479b9a9931a5"
format_stamp: "Formatted at 2026-08-06 18:45:06 on dist-test-slave-fgb3"
server_key: "d7b298422cbb79d85614786e9310dc38"
server_key_iv: "6a554d65ae5104d76be6f36f21c6131b"
server_key_version: "encryptionkey@0"
I20260806 18:45:06.220937 10403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "a9e0aaa7eb4346b89125479b9a9931a5"
format_stamp: "Formatted at 2026-08-06 18:45:06 on dist-test-slave-fgb3"
server_key: "d7b298422cbb79d85614786e9310dc38"
server_key_iv: "6a554d65ae5104d76be6f36f21c6131b"
server_key_version: "encryptionkey@0"
I20260806 18:45:06.225034 10403 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 18:45:06.227730 10425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:06.229146 10403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:45:06.229285 10403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "a9e0aaa7eb4346b89125479b9a9931a5"
format_stamp: "Formatted at 2026-08-06 18:45:06 on dist-test-slave-fgb3"
server_key: "d7b298422cbb79d85614786e9310dc38"
server_key_iv: "6a554d65ae5104d76be6f36f21c6131b"
server_key_version: "encryptionkey@0"
I20260806 18:45:06.229419 10403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:06.251425 10403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:06.254768 10403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:06.254964 10403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:06.255599 10403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:06.256481 10403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:06.256532 10403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:06.256585 10403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:06.256614 10403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:06.265208 10403 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:43263
I20260806 18:45:06.265295 10598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:43263 every 8 connection(s)
I20260806 18:45:06.266131 10403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:06.271572 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 10403
I20260806 18:45:06.271706 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:06.272004 2005 external_mini_cluster.cc:1428] Setting key fd98b268069153f27c3e5244b93af612
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:06.280589 9992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:06.268005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:39935 (local address 127.1.245.126:37305)
0806 18:45:06.268139 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:06.268142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:06.269028 (+ 886us) server_negotiation.cc:407] Connection header received
0806 18:45:06.270720 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:06.270724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:06.270777 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:06.270855 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:06.272528 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.273116 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.274283 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.274565 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.274653 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:06.277190 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:06.277193 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:06.277196 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:06.277224 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:06.278480 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.278992 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.278995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.278996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.279075 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.279442 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.279445 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.279446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.279677 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:06.279782 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:06.280341 (+ 559us) server_negotiation.cc:299] Negotiation successful
0806 18:45:06.280458 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:45:06.281404 10603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:06.268358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37305 (local address 127.1.245.67:39935)
0806 18:45:06.268849 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:06.268897 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:06.270493 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:06.271076 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:06.271086 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:06.271622 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:06.272360 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:06.272383 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.273243 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.273247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:06.274155 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:06.274166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.274723 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.274728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:06.274914 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:06.275523 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:06.275540 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:06.277023 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:06.278602 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:06.278607 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:06.278621 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:06.278898 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:06.279150 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:06.279153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:06.279154 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:06.279326 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:06.279798 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:06.279804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:06.280062 (+ 258us) client_negotiation.cc:769] Sending connection context
0806 18:45:06.280235 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 18:45:06.280550 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":123,"threads_started":1}
I20260806 18:45:06.282536 10600 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37305
I20260806 18:45:06.282758 10600 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:06.283285 10600 heartbeater.cc:511] Master 127.1.245.126:37305 requested a full tablet report, sending...
I20260806 18:45:06.284382 9934 ts_manager.cc:194] Registered new tserver with Master: a9e0aaa7eb4346b89125479b9a9931a5 (127.1.245.67:43263)
I20260806 18:45:06.285061 9934 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:39935
I20260806 18:45:06.286171 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/test-admin.keytab.
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:06.303388 2005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 18:45:06.306375 10611 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:06.304738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37240 (local address 127.1.245.65:34751)
0806 18:45:06.305043 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:06.305061 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:06.305085 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:45:06.305136 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:06.305141 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:06.305356 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:06.305474 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:06.306086 (+ 612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37240: BlockingRecv error: recv got EOF from 127.0.0.1:37240 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":119,"threads_started":1}
W20260806 18:45:06.306391 2005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:06.307418 2005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:06.308363 2005 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:45:06.318022 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:06.309266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37252 (local address 127.1.245.65:34751)
0806 18:45:06.309429 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:06.309434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:06.309451 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:45:06.309586 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:06.309590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:06.309639 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:06.309708 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:06.310333 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.311169 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.312088 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.312528 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.312623 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:06.314994 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:06.315004 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:06.315018 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:06.315044 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:06.316532 (+ 1488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.316910 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.316915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.316916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.316990 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.317176 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.317178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.317179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.317406 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:06.317564 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:06.317730 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 18:45:06.317864 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260806 18:45:06.321724 10611 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:06.320911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37256 (local address 127.1.245.65:34751)
0806 18:45:06.321051 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:06.321054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:06.321104 (+ 50us) server_negotiation.cc:407] Connection header received
0806 18:45:06.321193 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:06.321195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:06.321230 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:06.321319 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:06.321619 (+ 300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37256: BlockingRecv error: recv got EOF from 127.0.0.1:37256 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 18:45:06.321805 2005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:06.322772 2005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:06.323560 2005 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:45:06.331693 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:06.324298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.1.245.65:34751)
0806 18:45:06.324405 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:06.324409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:06.324479 (+ 70us) server_negotiation.cc:407] Connection header received
0806 18:45:06.324610 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:06.324613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:06.324655 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:06.324731 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:06.325140 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.325718 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.326539 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.326768 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.326854 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:06.328960 (+ 2106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:06.328963 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:06.328965 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:06.328987 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:06.330208 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.330646 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.330649 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.330650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.330737 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.330947 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.330949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.330950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.331113 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:06.331215 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:06.331406 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 18:45:06.331520 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 18:45:06.334255 10611 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:06.333399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.1.245.65:34751)
0806 18:45:06.333528 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:06.333532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:06.333548 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:45:06.333677 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:06.333700 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:06.333746 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:06.333814 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:06.334161 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37286: BlockingRecv error: recv got EOF from 127.0.0.1:37286 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260806 18:45:06.334472 2005 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786041874296929-2005-0/krb5kdc/krb5cc' of type FILE
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:06.335427 2005 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:06.336226 2005 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[9875](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:45:06.344472 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:06.336996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37294 (local address 127.1.245.65:34751)
0806 18:45:06.337090 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:06.337093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:06.337194 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:45:06.337279 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:06.337281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:06.337314 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:06.337389 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:06.337847 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.338473 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.339341 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:06.339604 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:06.339688 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:06.341873 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:06.341877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:06.341879 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:06.341903 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:06.343040 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.343409 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.343413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.343415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.343486 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:06.343719 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:06.343722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:06.343724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:06.343879 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:06.343991 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:06.344206 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 18:45:06.344348 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:45:06.345402 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 10023
I20260806 18:45:06.359663 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 10209
I20260806 18:45:06.369195 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 10403
I20260806 18:45:06.376798 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 9893
2026-08-06T18:45:06Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2291 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[10631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[10631](info): set up 2 sockets
Aug 06 18:45:06 dist-test-slave-fgb3 krb5kdc[10631](info): commencing operation
krb5kdc: starting...
W20260806 18:45:07.522675 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.120s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:45:07 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:45:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:07Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.126.keytab.
Entry for principal oryx/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.126.keytab.
I20260806 18:45:07.661895 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:37393
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.126.keytab
--principal=oryx/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:37393
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:07.777100 10649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:07.777417 10649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:07.777462 10649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:07.781647 10649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:07.781713 10649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:07.781731 10649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:07.781747 10649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:07.781761 10649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:07.786968 10649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:37393
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.126.keytab
--principal=oryx/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:37393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:07.788017 10649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:07.789006 10649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:07.797339 10659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:07.797392 10657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:07.797492 10654 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:07.797576 10649 server_base.cc:1034] running on GCE node
I20260806 18:45:07.798183 10649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:07.799293 10649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:07.800457 10649 hybrid_clock.cc:648] HybridClock initialized: now 1786041907800430 us; error 37 us; skew 500 ppm
Aug 06 18:45:07 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041907, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:07.803157 10649 init.cc:377] Logged in from keytab as oryx/127.1.245.126@KRBTEST.COM (short username oryx)
I20260806 18:45:07.804028 10649 webserver.cc:460] Webserver started at http://127.1.245.126:41095/ using document root <none> and password file <none>
I20260806 18:45:07.804580 10649 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:07.804644 10649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:07.804834 10649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:07.806841 10649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "246273a6f7bc49578d7729671bee2816"
format_stamp: "Formatted at 2026-08-06 18:45:07 on dist-test-slave-fgb3"
server_key: "a8043e7cec4f5502b067575bff291d02"
server_key_iv: "613249434f5d8e857292ef74834fcd1a"
server_key_version: "encryptionkey@0"
I20260806 18:45:07.807377 10649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "246273a6f7bc49578d7729671bee2816"
format_stamp: "Formatted at 2026-08-06 18:45:07 on dist-test-slave-fgb3"
server_key: "a8043e7cec4f5502b067575bff291d02"
server_key_iv: "613249434f5d8e857292ef74834fcd1a"
server_key_version: "encryptionkey@0"
I20260806 18:45:07.810897 10649 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:45:07.813256 10670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:07.814477 10649 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:07.814589 10649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "246273a6f7bc49578d7729671bee2816"
format_stamp: "Formatted at 2026-08-06 18:45:07 on dist-test-slave-fgb3"
server_key: "a8043e7cec4f5502b067575bff291d02"
server_key_iv: "613249434f5d8e857292ef74834fcd1a"
server_key_version: "encryptionkey@0"
I20260806 18:45:07.814675 10649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:07.833230 10649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:07.836222 10649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:07.836413 10649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:07.843103 10649 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:37393
I20260806 18:45:07.843372 10745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:37393 every 8 connection(s)
I20260806 18:45:07.844031 10649 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:07.846850 10746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:07.849016 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 10649
I20260806 18:45:07.849145 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:07.849442 2005 external_mini_cluster.cc:1428] Setting key 822e1456c6657f289a4d7d71d5033728
I20260806 18:45:07.853179 10746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: Bootstrap starting.
I20260806 18:45:07.855738 10746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:07.856521 10746 log.cc:824] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:45:07 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:07.858865 10746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: No bootstrap required, opened a new log
I20260806 18:45:07.863154 10746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } }
I20260806 18:45:07.863392 10746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:07.863453 10746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246273a6f7bc49578d7729671bee2816, State: Initialized, Role: FOLLOWER
I20260806 18:45:07.863827 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:07.850884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.1.245.126:37393)
0806 18:45:07.851298 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:07.851315 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:07.851352 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:45:07.852158 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:07.852180 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:07.852551 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:07.852839 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:07.853758 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:07.854844 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:07.855956 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:07.856449 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:07.856571 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:07.859445 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:07.859451 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:07.859464 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:07.859492 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:07.861098 (+ 1606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:07.861573 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:07.861578 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:07.861582 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:07.861682 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:07.861939 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:07.861941 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:07.861942 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:07.862344 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:07.862507 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:07.862772 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 18:45:07.863045 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":116,"threads_started":1}
I20260806 18:45:07.863893 10746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } }
I20260806 18:45:07.864061 10746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:07.864122 10746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:07.864221 10746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:07.865097 10746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } }
I20260806 18:45:07.865432 10746 leader_election.cc:302] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 246273a6f7bc49578d7729671bee2816; no voters:
I20260806 18:45:07.865738 10746 leader_election.cc:288] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:07.865895 10754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:07.866089 10754 raft_consensus.cc:686] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 1 LEADER]: Becoming Leader. State: Replica: 246273a6f7bc49578d7729671bee2816, State: Running, Role: LEADER
I20260806 18:45:07.866466 10754 consensus_queue.cc:237] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } }
I20260806 18:45:07.867045 10746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:07.868096 10756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 246273a6f7bc49578d7729671bee2816. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } }
I20260806 18:45:07.868232 10756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:07.868528 10755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } }
I20260806 18:45:07.868618 10755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:07.868754 10765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:07.871707 10765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:07.877530 10765 catalog_manager.cc:1347] Generated new cluster ID: 04423f5d736947abb745a66cce39e2f5
I20260806 18:45:07.877604 10765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:07.923398 10765 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:07.924412 10765 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:07.938233 10765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: Generated new TSK 0
I20260806 18:45:07.939082 10765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab.
Entry for principal oryx/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab.
I20260806 18:45:07.970248 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37393
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab
--principal=oryx/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:08.084465 10783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:08.084736 10783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:08.084779 10783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:08.088841 10783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:08.088898 10783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:08.088963 10783 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:08.094125 10783 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab
--principal=oryx/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:08.095168 10783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:08.096163 10783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:08.105635 10792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:08.105796 10783 server_base.cc:1034] running on GCE node
W20260806 18:45:08.105616 10795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.105623 10793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:08.106281 10783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:08.106981 10783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:08.108156 10783 hybrid_clock.cc:648] HybridClock initialized: now 1786041908108137 us; error 30 us; skew 500 ppm
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:08.110916 10783 init.cc:377] Logged in from keytab as oryx/127.1.245.65@KRBTEST.COM (short username oryx)
I20260806 18:45:08.111794 10783 webserver.cc:460] Webserver started at http://127.1.245.65:35283/ using document root <none> and password file <none>
I20260806 18:45:08.112372 10783 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:08.112442 10783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:08.112648 10783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:08.114746 10783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "ea17dfec40434cd88bc6e868eb83fa32"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "1230ba912bf9e851bb990bbc18783751"
server_key_iv: "ed6931851151249ea645ccab855d73ea"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.115286 10783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "ea17dfec40434cd88bc6e868eb83fa32"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "1230ba912bf9e851bb990bbc18783751"
server_key_iv: "ed6931851151249ea645ccab855d73ea"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.119390 10783 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:45:08.121776 10807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.123020 10783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:45:08.123139 10783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "ea17dfec40434cd88bc6e868eb83fa32"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "1230ba912bf9e851bb990bbc18783751"
server_key_iv: "ed6931851151249ea645ccab855d73ea"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.123238 10783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:08.141535 10783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:08.144765 10783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:08.144963 10783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:08.145584 10783 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:08.146476 10783 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:08.146528 10783 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.146584 10783 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:08.146613 10783 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.155164 10783 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:41519
I20260806 18:45:08.155297 10966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:41519 every 8 connection(s)
I20260806 18:45:08.156092 10783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:08.157629 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 10783
I20260806 18:45:08.157734 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:45:08.158102 2005 external_mini_cluster.cc:1428] Setting key 381a90bb01d3c27b91b3219632521d7b
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:45:08.170219 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.157877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:40559 (local address 127.1.245.126:37393)
0806 18:45:08.158063 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.158067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.159228 (+ 1161us) server_negotiation.cc:407] Connection header received
0806 18:45:08.160292 (+ 1064us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.160294 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.160344 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.160413 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.161917 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.162531 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.163653 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.163884 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.163971 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.166442 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.166446 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.166448 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.166470 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.167911 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.168556 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.168560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.168562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.168645 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.169024 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.169027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.169029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.169235 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.169318 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.169963 (+ 645us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.170088 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.66@KRBTEST.COM" created.
I20260806 18:45:08.171258 10970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.158342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.65:40559)
0806 18:45:08.159088 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.159131 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.160039 (+ 908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.160596 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.160605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.161087 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.161760 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.161781 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.162647 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.162650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.163547 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.163560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.164023 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.164027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.164227 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.164870 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.164886 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.166274 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.168041 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.168048 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.168064 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.168443 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.168757 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.168761 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.168763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.168932 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.169460 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.169468 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.169725 (+ 257us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.169897 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.170160 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":137,"threads_started":1}
I20260806 18:45:08.172598 10968 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37393
I20260806 18:45:08.172873 10968 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:08.173389 10968 heartbeater.cc:511] Master 127.1.245.126:37393 requested a full tablet report, sending...
I20260806 18:45:08.174957 10694 ts_manager.cc:194] Registered new tserver with Master: ea17dfec40434cd88bc6e868eb83fa32 (127.1.245.65:41519)
I20260806 18:45:08.176168 10694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:40559
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab.
Entry for principal oryx/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab.
I20260806 18:45:08.189123 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37393
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab
--principal=oryx/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:08.302675 10973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:08.302928 10973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:08.302973 10973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:08.307103 10973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:08.307171 10973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:08.307230 10973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:08.312742 10973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab
--principal=oryx/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:08.313874 10973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:08.314973 10973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:08.324862 10984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.324892 10985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.324872 10987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:08.325028 10973 server_base.cc:1034] running on GCE node
I20260806 18:45:08.325590 10973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:08.326236 10973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:08.327390 10973 hybrid_clock.cc:648] HybridClock initialized: now 1786041908327379 us; error 20 us; skew 500 ppm
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:08.330204 10973 init.cc:377] Logged in from keytab as oryx/127.1.245.66@KRBTEST.COM (short username oryx)
I20260806 18:45:08.331158 10973 webserver.cc:460] Webserver started at http://127.1.245.66:35799/ using document root <none> and password file <none>
I20260806 18:45:08.331699 10973 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:08.331758 10973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:08.331975 10973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:08.334054 10973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "76a9cf82ac3e485eb75e421bd6565b59"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "acb7576fa11691726696f37625e9eabc"
server_key_iv: "4521883f28164701bbea65dc3a5683cf"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.334611 10973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "76a9cf82ac3e485eb75e421bd6565b59"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "acb7576fa11691726696f37625e9eabc"
server_key_iv: "4521883f28164701bbea65dc3a5683cf"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.338346 10973 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:45:08.340816 10997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.342149 10973 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:08.342268 10973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "76a9cf82ac3e485eb75e421bd6565b59"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "acb7576fa11691726696f37625e9eabc"
server_key_iv: "4521883f28164701bbea65dc3a5683cf"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.342411 10973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:08.367808 10973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:08.371338 10973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:08.371630 10973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:08.372328 10973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:08.373260 10973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:08.373319 10973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.373378 10973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:08.373404 10973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.382022 10973 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:37589
I20260806 18:45:08.382094 11154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:37589 every 8 connection(s)
I20260806 18:45:08.383003 10973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:08.386458 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 10973
I20260806 18:45:08.386554 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:45:08.386822 2005 external_mini_cluster.cc:1428] Setting key 869d7d458b3cbb584cbcd95c0fc3c096
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:45:08.396912 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.384924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:57827 (local address 127.1.245.126:37393)
0806 18:45:08.385064 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.385067 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.386184 (+ 1117us) server_negotiation.cc:407] Connection header received
0806 18:45:08.387150 (+ 966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.387154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.387212 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.387299 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.388624 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.389143 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.390267 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.390556 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.390647 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.393268 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.393273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.393276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.393307 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.394924 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.395397 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.395402 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.395404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.395498 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.395784 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.395788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.395790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.396011 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.396126 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.396616 (+ 490us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.396744 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:45:08.397640 11158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.385298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.66:57827)
0806 18:45:08.386018 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.386058 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.386946 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.387643 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.387649 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.388012 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.388483 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.388496 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.389261 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.389264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.390139 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.390149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.390695 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.390700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.390871 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.391431 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.391451 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.393094 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.395030 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.395035 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.395048 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.395293 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.395576 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.395578 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.395579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.395699 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.396108 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.396113 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.396373 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.396526 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.396774 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":123,"threads_started":1}
I20260806 18:45:08.398854 11155 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37393
I20260806 18:45:08.399080 11155 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:08.399576 11155 heartbeater.cc:511] Master 127.1.245.126:37393 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.67@KRBTEST.COM" created.
I20260806 18:45:08.400741 10694 ts_manager.cc:194] Registered new tserver with Master: 76a9cf82ac3e485eb75e421bd6565b59 (127.1.245.66:37589)
I20260806 18:45:08.401363 10694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:57827
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab.
Entry for principal oryx/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab.
I20260806 18:45:08.415871 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37393
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab
--principal=oryx/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:08.528079 11165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:08.528347 11165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:08.528393 11165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:08.532404 11165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:08.532459 11165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:08.532529 11165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:08.537654 11165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab
--principal=oryx/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:37393
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:08.538744 11165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:08.539875 11165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:08.549836 11173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.549870 11171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.549870 11175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:08.550057 11165 server_base.cc:1034] running on GCE node
I20260806 18:45:08.550570 11165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:08.551214 11165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:08.552377 11165 hybrid_clock.cc:648] HybridClock initialized: now 1786041908552344 us; error 43 us; skew 500 ppm
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:08.555105 11165 init.cc:377] Logged in from keytab as oryx/127.1.245.67@KRBTEST.COM (short username oryx)
I20260806 18:45:08.556021 11165 webserver.cc:460] Webserver started at http://127.1.245.67:34517/ using document root <none> and password file <none>
I20260806 18:45:08.556569 11165 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:08.556631 11165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:08.556838 11165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:08.558975 11165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "3fdb241e0e474cb3b1534857c34eaec7"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "fb1a4b25ad794683de0d4e2e1098605a"
server_key_iv: "6bbc0dac1e89a696c445199dc4397dc4"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.559537 11165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "3fdb241e0e474cb3b1534857c34eaec7"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "fb1a4b25ad794683de0d4e2e1098605a"
server_key_iv: "6bbc0dac1e89a696c445199dc4397dc4"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.563429 11165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:45:08.565979 11186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.567409 11165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:45:08.567531 11165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "3fdb241e0e474cb3b1534857c34eaec7"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "fb1a4b25ad794683de0d4e2e1098605a"
server_key_iv: "6bbc0dac1e89a696c445199dc4397dc4"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.567668 11165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:08.592298 11165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:08.595798 11165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:08.596005 11165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:08.596669 11165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:08.597609 11165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:08.597661 11165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.597719 11165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:08.597748 11165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.607244 11165 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:34405
I20260806 18:45:08.607254 11342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:34405 every 8 connection(s)
I20260806 18:45:08.608197 11165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:08.613250 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 11165
I20260806 18:45:08.613405 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:08.613718 2005 external_mini_cluster.cc:1428] Setting key d130610f87536ca9f42764043ab24a70
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:08.623095 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.610281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:60329 (local address 127.1.245.126:37393)
0806 18:45:08.610503 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.610509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.611724 (+ 1215us) server_negotiation.cc:407] Connection header received
0806 18:45:08.612622 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.612625 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.612676 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.612758 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.614385 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.615117 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.616569 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.616798 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.616880 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.619444 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.619447 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.619449 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.619478 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.620974 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.621515 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.621519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.621520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.621607 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.621892 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.621894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.621895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.622076 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.622150 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.622828 (+ 678us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.622963 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260806 18:45:08.623929 11345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.610833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.67:60329)
0806 18:45:08.611565 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.611610 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.612424 (+ 814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.612971 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.612980 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.613547 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.614197 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.614216 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.615253 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.615256 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.616450 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.616460 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.616918 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.616921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.617061 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.617660 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.617677 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.619264 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.621108 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.621114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.621129 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.621426 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.621687 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.621689 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.621690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.621825 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.622224 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.622230 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.622584 (+ 354us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.622765 (+ 181us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.623052 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":125,"threads_started":1}
I20260806 18:45:08.625198 11343 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37393
I20260806 18:45:08.625447 11343 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:08.625944 11343 heartbeater.cc:511] Master 127.1.245.126:37393 requested a full tablet report, sending...
I20260806 18:45:08.627112 10693 ts_manager.cc:194] Registered new tserver with Master: 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405)
I20260806 18:45:08.627882 10693 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:60329
I20260806 18:45:08.628120 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.245.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.125.keytab.
Entry for principal oryx/127.1.245.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.125.keytab.
I20260806 18:45:08.657478 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.125:35987
--webserver_interface=127.1.245.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.125.keytab
--principal=oryx/127.1.245.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:37393,127.1.245.125:35987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:08.772003 11348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:08.772279 11348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:08.772320 11348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:08.776351 11348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:08.776410 11348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:08.776429 11348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:08.776450 11348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:08.776471 11348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:08.781836 11348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:37393,127.1.245.125:35987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.125.keytab
--principal=oryx/127.1.245.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.125:35987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.245.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
W20260806 18:45:08.782047 11348 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.245.126:37393,127.1.245.125:35987'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 18:45:08.783006 11348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:08.784024 11348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:08.792474 11358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.792495 11361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.792520 11359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:08.792836 11348 server_base.cc:1034] running on GCE node
I20260806 18:45:08.793439 11348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:08.794589 11348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:08.795753 11348 hybrid_clock.cc:648] HybridClock initialized: now 1786041908795715 us; error 49 us; skew 500 ppm
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:08.798486 11348 init.cc:377] Logged in from keytab as oryx/127.1.245.125@KRBTEST.COM (short username oryx)
I20260806 18:45:08.799440 11348 webserver.cc:460] Webserver started at http://127.1.245.125:45093/ using document root <none> and password file <none>
I20260806 18:45:08.799975 11348 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:08.800034 11348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:08.800213 11348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:08.802241 11348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data/instance:
uuid: "c77cb7e640c048a0bc093a588c5ed78d"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "cb3c1e1b395d995d5db8c68e7ebef301"
server_key_iv: "b1c5a8bc36c93f4b1f0a267817b9ce29"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.802829 11348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal/instance:
uuid: "c77cb7e640c048a0bc093a588c5ed78d"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "cb3c1e1b395d995d5db8c68e7ebef301"
server_key_iv: "b1c5a8bc36c93f4b1f0a267817b9ce29"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.806720 11348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:45:08.809245 11373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.810514 11348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:08.810637 11348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
uuid: "c77cb7e640c048a0bc093a588c5ed78d"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "cb3c1e1b395d995d5db8c68e7ebef301"
server_key_iv: "b1c5a8bc36c93f4b1f0a267817b9ce29"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.810739 11348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:08.823711 11348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:08.827047 11348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:08.827231 11348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:08.833953 11348 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.125:35987
I20260806 18:45:08.834251 11449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.125:35987 every 8 connection(s)
I20260806 18:45:08.835017 11348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
I20260806 18:45:08.837702 11450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:08.841506 11450 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:45:08.844475 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 11348
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.125@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.245.125@KRBTEST.COM
I20260806 18:45:08.855718 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.843405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34556 (local address 127.1.245.126:37393)
0806 18:45:08.843544 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.843549 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.844359 (+ 810us) server_negotiation.cc:407] Connection header received
0806 18:45:08.845399 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.845402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.845458 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.845529 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.846729 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.847248 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.848764 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.848983 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.849073 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.851653 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.851656 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.851659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.851688 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.853307 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.853853 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.853857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.853859 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.853937 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.854307 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.854337 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.854339 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.854590 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.854679 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.855442 (+ 763us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.855562 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:45:08.856536 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.843743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:34556)
0806 18:45:08.844200 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.844262 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.845187 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.845695 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.845703 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.846047 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.846599 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.846612 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.847369 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.847372 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.848652 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.848660 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.849125 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.849129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.849338 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.849943 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.849962 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.851510 (+ 1548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.853417 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.853423 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.853436 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.853763 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.854032 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.854036 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.854038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.854212 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.854730 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.854737 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.855022 (+ 285us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.855378 (+ 356us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.855694 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":108,"threads_started":1}
I20260806 18:45:08.858017 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.847482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58898 (local address 127.1.245.125:35987)
0806 18:45:08.847852 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.847863 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.847965 (+ 102us) server_negotiation.cc:407] Connection header received
0806 18:45:08.848072 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.848077 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.848254 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.848409 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.849205 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.850145 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.851306 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.851740 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.851832 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.854405 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.854414 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.854427 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.854450 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.856027 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.856422 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.856427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.856429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.856504 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.856762 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.856766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.856767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.857017 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.857222 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.857710 (+ 488us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.857852 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":85,"threads_started":1}
I20260806 18:45:08.858073 11450 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.125@KRBTEST.COM for oryx/127.1.245.125@KRBTEST.COM
I20260806 18:45:08.865075 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.858539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.0.0.1:58906)
0806 18:45:08.858646 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.858659 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.858723 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.859029 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.859031 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.859163 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.859239 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.859244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.859978 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.859981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.860458 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.860466 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.860906 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.860909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.861015 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.861373 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.861383 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.862486 (+ 1103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.864000 (+ 1514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.864002 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.864004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.864170 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.864384 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.864386 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.864387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.864446 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.864699 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.864701 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.864767 (+ 66us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.864856 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.864942 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":2}
I20260806 18:45:08.865108 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.858803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58906 (local address 127.1.245.125:35987)
0806 18:45:08.858886 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.858889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.858896 (+ 7us) server_negotiation.cc:407] Connection header received
0806 18:45:08.858916 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.858918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.858945 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.859000 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.859347 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.859888 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.860555 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.860816 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.860900 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.862614 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.862617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.862619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.862640 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.863902 (+ 1262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.864243 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.864246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.864248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.864320 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.864502 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.864503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.864504 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.864642 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.864707 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.864901 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.864986 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":34}
I20260806 18:45:08.868189 11450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Bootstrap starting.
I20260806 18:45:08.870489 11450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:08.871269 11450 log.cc:824] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:08.873164 11450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: No bootstrap required, opened a new log
I20260806 18:45:08.876240 11450 raft_consensus.cc:348] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } }
I20260806 18:45:08.876441 11450 raft_consensus.cc:374] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:08.876493 11450 raft_consensus.cc:729] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c77cb7e640c048a0bc093a588c5ed78d, State: Initialized, Role: FOLLOWER
I20260806 18:45:08.876936 11450 consensus_queue.cc:260] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } }
I20260806 18:45:08.877403 11457 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } } }
I20260806 18:45:08.877540 11457 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:45:08.878151 11450 sys_catalog.cc:564] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:45:08.882611 11471 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:45:08.882690 11471 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:45:08.884608 11348 master_runner.cc:428] Detected that this master c77cb7e640c048a0bc093a588c5ed78d is joining an existing cluster
I20260806 18:45:08.884672 11348 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.245.125:35987
I20260806 18:45:08.894488 11485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.886989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.0.0.1:58916)
0806 18:45:08.887338 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.887351 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.887449 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.887765 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.887767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.887973 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.888135 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.888142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.889080 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.889083 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.889654 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.889661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.890129 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.890132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.890210 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.890680 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.890695 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.891016 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.892882 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.892886 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.892889 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.893120 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.893403 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.893406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.893408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.893494 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.893887 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.893890 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.893984 (+ 94us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.894102 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.894246 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":22,"thread_start_us":82,"threads_started":1}
I20260806 18:45:08.894502 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.887288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58916 (local address 127.1.245.125:35987)
0806 18:45:08.887392 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.887395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.887407 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:45:08.887554 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.887557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.887594 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.887657 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.888248 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.888967 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.889772 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.890026 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.890111 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.891122 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.891126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.891128 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.891154 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.892763 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.893217 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.893220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.893222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.893314 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.893567 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.893570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.893572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.893797 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.893889 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.894154 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.894337 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":54}
I20260806 18:45:08.895555 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.886944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34562 (local address 127.1.245.126:37393)
0806 18:45:08.887067 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.887071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.887340 (+ 269us) server_negotiation.cc:407] Connection header received
0806 18:45:08.887528 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.887532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.887583 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.887649 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.888123 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.888943 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.890181 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.890558 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.890635 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.891858 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.891862 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.891864 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.891902 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.893318 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.893803 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.893806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.893808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.893890 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.894181 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.894184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.894186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.894416 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.894490 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.895298 (+ 808us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.895415 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:45:08.896008 11482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.886869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:34562)
0806 18:45:08.887247 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.887269 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.887414 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.887674 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.887676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.887883 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.888009 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.888016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.889063 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.889066 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.890038 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.890049 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.890787 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.890791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.890882 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.891325 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.891337 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.891720 (+ 383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.893568 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.893570 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.893571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.893723 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.893999 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.894002 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.894004 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.894086 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.895016 (+ 930us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.895019 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.895105 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.895778 (+ 673us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.895884 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":8,"thread_start_us":96,"threads_started":1}
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041907, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.126@KRBTEST.COM for oryx/127.1.245.125@KRBTEST.COM
I20260806 18:45:08.908046 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.900135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58932 (local address 127.1.245.125:35987)
0806 18:45:08.900230 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.900234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.900515 (+ 281us) server_negotiation.cc:407] Connection header received
0806 18:45:08.900684 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.900688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.900735 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.900795 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.901401 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.901885 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.902741 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.902955 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.903037 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.905225 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.905229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.905231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.905254 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.906499 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.906908 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.906911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.906913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.906984 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.907227 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.907230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.907232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.907398 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.907485 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.907795 (+ 310us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.907896 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 18:45:08.908070 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.900094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.0.0.1:58932)
0806 18:45:08.900406 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.900448 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.900570 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.900841 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.900847 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.901185 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.901295 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.901301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.901994 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.901997 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.902639 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.902648 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.903048 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.903052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.903165 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.903581 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.903593 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.905102 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.906592 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.906601 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.906607 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.906820 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.907062 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.907065 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.907067 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.907143 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.907499 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.907506 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.907615 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.907733 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.907912 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":93,"threads_started":1}
I20260806 18:45:08.909822 10693 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.245.125:35987: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d"
member_type: NON_VOTER
last_known_addr {
host: "127.1.245.125"
port: 35987
}
attrs {
promote: true
}
}
dest_uuid: "246273a6f7bc49578d7729671bee2816"
cas_config_opid_index: -1
I20260806 18:45:08.910638 10693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } }
I20260806 18:45:08.912420 11488 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER c77cb7e640c048a0bc093a588c5ed78d (127.1.245.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } }
I20260806 18:45:08.913169 11412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:08.914225 11488 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.245.125:35987
I20260806 18:45:08.914399 11412 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Refusing update from remote peer 246273a6f7bc49578d7729671bee2816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 18:45:08.914461 11488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } }
I20260806 18:45:08.914678 11488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:08.914978 11348 master.cc:537] Master@127.1.245.125:35987 shutting down...
I20260806 18:45:08.915508 11490 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:45:08.918076 10671 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 18:45:08.919608 10671 consensus_peers.cc:487] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 -> Peer c77cb7e640c048a0bc093a588c5ed78d (127.1.245.125:35987): Couldn't send request to peer c77cb7e640c048a0bc093a588c5ed78d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:45:08.920048 11348 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:08.920300 11348 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:08.920397 11348 tablet_replica.cc:330] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: stopping tablet replica
WARNING: no policy specified for oryx/127.1.245.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.124@KRBTEST.COM" created.
I20260806 18:45:08.935806 11348 master.cc:552] Master@127.1.245.125:35987 shutdown complete.
I20260806 18:45:08.935858 11348 master_runner.cc:305] Clearing existing system tablet
I20260806 18:45:08.936082 11348 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:08.936132 11348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:08.937531 11348 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 18:45:08.938414 11498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.938655 11348 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:08.938769 11348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:08.938807 11348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
uuid: "c77cb7e640c048a0bc093a588c5ed78d"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "cb3c1e1b395d995d5db8c68e7ebef301"
server_key_iv: "b1c5a8bc36c93f4b1f0a267817b9ce29"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.940687 11348 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.124.keytab.
Entry for principal oryx/127.1.245.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.124.keytab.
I20260806 18:45:08.942045 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.124:45209
--webserver_interface=127.1.245.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.124.keytab
--principal=oryx/127.1.245.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:37393,127.1.245.125:35987,127.1.245.124:45209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:45:08.942994 11348 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:45:08.943084 11348 log.cc:1196] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 18:45:08.943312 11348 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Deleting consensus metadata
I20260806 18:45:08.943495 11348 master_runner.cc:315] Copying system tablet from 127.1.245.126:37393
I20260806 18:45:08.944916 11348 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: tablet copy: Beginning tablet copy session from remote peer at address 127.1.245.126:37393
I20260806 18:45:08.966687 11506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.954708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:34572)
0806 18:45:08.958520 (+ 3812us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:08.958538 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:08.958689 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:08.958995 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:08.958999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:08.959216 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:08.959367 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.959374 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.960452 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.960456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.961368 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:08.961379 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.961975 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.961979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:08.962073 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.962664 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:08.962680 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:08.963099 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:08.965008 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.965012 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.965014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.965276 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.965566 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:08.965569 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:08.965570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:08.965660 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:08.966053 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:08.966056 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:08.966159 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 18:45:08.966271 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:45:08.966448 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3717,"thread_start_us":94,"threads_started":1}
I20260806 18:45:08.966841 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:08.954813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34572 (local address 127.1.245.126:37393)
0806 18:45:08.954953 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:08.954958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:08.958654 (+ 3696us) server_negotiation.cc:407] Connection header received
0806 18:45:08.958789 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:08.958793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:08.958854 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:08.958952 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:08.959492 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.960340 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.961496 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:08.961849 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:08.961959 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:08.963259 (+ 1300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:08.963263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:08.963266 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:08.963296 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:08.964896 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.965371 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.965375 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.965377 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.965481 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:08.965740 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:08.965743 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:08.965745 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:08.965962 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:08.966059 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:08.966384 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 18:45:08.966607 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:45:08.967860 10722 tablet_copy_service.cc:140] P 246273a6f7bc49578d7729671bee2816: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c77cb7e640c048a0bc093a588c5ed78d ({username='oryx', principal='oryx/127.1.245.125@KRBTEST.COM'} at 127.0.0.1:34572)
I20260806 18:45:08.968045 10722 tablet_copy_service.cc:161] P 246273a6f7bc49578d7729671bee2816: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c77cb7e640c048a0bc093a588c5ed78d at {username='oryx', principal='oryx/127.1.245.125@KRBTEST.COM'} at 127.0.0.1:34572: session id = c77cb7e640c048a0bc093a588c5ed78d-00000000000000000000000000000000
I20260806 18:45:08.971030 10722 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:45:08.973127 11348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:45:08.975859 11348 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: tablet copy: Starting download of 0 data blocks...
I20260806 18:45:08.976174 11348 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: tablet copy: Starting download of 1 WAL segments...
I20260806 18:45:08.978206 11348 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:45:08.980337 10722 tablet_copy_service.cc:342] P 246273a6f7bc49578d7729671bee2816: Request end of tablet copy session c77cb7e640c048a0bc093a588c5ed78d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.245.125@KRBTEST.COM'} at 127.0.0.1:34572
I20260806 18:45:08.980484 10722 tablet_copy_service.cc:433] P 246273a6f7bc49578d7729671bee2816: ending tablet copy session c77cb7e640c048a0bc093a588c5ed78d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c77cb7e640c048a0bc093a588c5ed78d
I20260806 18:45:08.988689 11348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:08.990679 11507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.990703 11510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:08.991183 11508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:08.991374 11348 server_base.cc:1034] running on GCE node
I20260806 18:45:08.991532 11348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:08.991829 11348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:08.992967 11348 hybrid_clock.cc:648] HybridClock initialized: now 1786041908992952 us; error 32 us; skew 500 ppm
Aug 06 18:45:08 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041908, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:08.994617 11348 init.cc:377] Logged in from keytab as oryx/127.1.245.125@KRBTEST.COM (short username oryx)
I20260806 18:45:08.995112 11348 webserver.cc:460] Webserver started at http://127.1.245.125:45093/ using document root <none> and password file <none>
I20260806 18:45:08.995311 11348 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:08.995348 11348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:08.996824 11348 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:08.997905 11522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:08.998239 11348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:08.998370 11348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
uuid: "c77cb7e640c048a0bc093a588c5ed78d"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "cb3c1e1b395d995d5db8c68e7ebef301"
server_key_iv: "b1c5a8bc36c93f4b1f0a267817b9ce29"
server_key_version: "encryptionkey@0"
I20260806 18:45:08.998471 11348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:09.021587 11348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:09.023840 11348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:09.028978 11348 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.125:35987
I20260806 18:45:09.029676 11348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
I20260806 18:45:09.031653 11594 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:45:09.032756 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Bootstrap starting.
I20260806 18:45:09.034449 11592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.125:35987 every 8 connection(s)
I20260806 18:45:09.045014 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:45:09.045352 11594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: Bootstrap complete.
I20260806 18:45:09.046113 11594 raft_consensus.cc:348] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } }
I20260806 18:45:09.046267 11594 raft_consensus.cc:729] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c77cb7e640c048a0bc093a588c5ed78d, State: Initialized, Role: LEARNER
I20260806 18:45:09.046422 11594 consensus_queue.cc:260] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } }
I20260806 18:45:09.046866 11597 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } }
I20260806 18:45:09.046976 11597 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: This master's current role is: LEARNER
I20260806 18:45:09.047024 11594 sys_catalog.cc:564] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:09.050222 11614 catalog_manager.cc:1259] Loaded cluster ID: 04423f5d736947abb745a66cce39e2f5
I20260806 18:45:09.050285 11614 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: loading cluster ID for follower catalog manager: success
I20260806 18:45:09.051770 11614 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: acquiring CA information for follower catalog manager: success
I20260806 18:45:09.052822 11614 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 18:45:09.077631 11499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:09.077922 11499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:09.078001 11499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:09.082007 11499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:09.082070 11499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:09.082098 11499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:09.082118 11499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:09.082144 11499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:09.087952 11499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:37393,127.1.245.125:35987,127.1.245.124:45209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.124.keytab
--principal=oryx/127.1.245.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.124:45209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.245.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:09.089138 11499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:09.090377 11499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:09.099093 11624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:09.099086 11621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.099154 11499 server_base.cc:1034] running on GCE node
W20260806 18:45:09.099086 11620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.099931 11499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:09.101009 11499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:09.102178 11499 hybrid_clock.cc:648] HybridClock initialized: now 1786041909102161 us; error 28 us; skew 500 ppm
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:09.105012 11499 init.cc:377] Logged in from keytab as oryx/127.1.245.124@KRBTEST.COM (short username oryx)
I20260806 18:45:09.105932 11499 webserver.cc:460] Webserver started at http://127.1.245.124:37323/ using document root <none> and password file <none>
I20260806 18:45:09.106526 11499 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:09.106595 11499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:09.106805 11499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:09.108843 11499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data/instance:
uuid: "b9304d77916a4a1bacb81ddad68f9845"
format_stamp: "Formatted at 2026-08-06 18:45:09 on dist-test-slave-fgb3"
server_key: "7daab170dbc2e72930eda8b2dd50dcf3"
server_key_iv: "38f8dd300f89f9ca917f18c931cee33d"
server_key_version: "encryptionkey@0"
I20260806 18:45:09.109400 11499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal/instance:
uuid: "b9304d77916a4a1bacb81ddad68f9845"
format_stamp: "Formatted at 2026-08-06 18:45:09 on dist-test-slave-fgb3"
server_key: "7daab170dbc2e72930eda8b2dd50dcf3"
server_key_iv: "38f8dd300f89f9ca917f18c931cee33d"
server_key_version: "encryptionkey@0"
I20260806 18:45:09.113008 11499 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:45:09.115446 11633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.116683 11499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:09.116799 11499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
uuid: "b9304d77916a4a1bacb81ddad68f9845"
format_stamp: "Formatted at 2026-08-06 18:45:09 on dist-test-slave-fgb3"
server_key: "7daab170dbc2e72930eda8b2dd50dcf3"
server_key_iv: "38f8dd300f89f9ca917f18c931cee33d"
server_key_version: "encryptionkey@0"
I20260806 18:45:09.116894 11499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:09.127290 11499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:09.130293 11499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:09.130520 11499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:09.137492 11499 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.124:45209
I20260806 18:45:09.137751 11718 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.124:45209 every 8 connection(s)
I20260806 18:45:09.138543 11499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
I20260806 18:45:09.139649 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 11499
I20260806 18:45:09.141774 11719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:09.146600 11719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.245.124@KRBTEST.COM
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.124@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:09.155218 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.141103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.1.245.124:45209)
0806 18:45:09.141544 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.141564 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.141608 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:45:09.142505 (+ 897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.142529 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.142959 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.143319 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.143998 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.145511 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.146448 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.147218 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.147387 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.150215 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.150220 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.150229 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.150256 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.152087 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.152490 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.152496 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.152503 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.152621 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.152950 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.152953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.152955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.153521 (+ 566us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.153727 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.153963 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.154247 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"spinlock_wait_cycles":4352,"thread_start_us":116,"threads_started":1}
I20260806 18:45:09.158571 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.148708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34576 (local address 127.1.245.126:37393)
0806 18:45:09.148850 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.148854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.149693 (+ 839us) server_negotiation.cc:407] Connection header received
0806 18:45:09.149845 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.149847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.149895 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.149959 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.150731 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.151395 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.152496 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.152737 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.152849 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.155199 (+ 2350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.155203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.155206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.155238 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.156735 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.157260 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.157264 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.157266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.157359 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.157654 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.157657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.157659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.157848 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.157937 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.158269 (+ 332us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.158413 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:45:09.158605 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.148514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:34576)
0806 18:45:09.149534 (+ 1020us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.149589 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.149724 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.150047 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.150052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.150474 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.150604 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.150610 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.151509 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.151512 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.152383 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.152391 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.152856 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.152858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.152959 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.153462 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.153476 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.155028 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.156869 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.156877 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.156882 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.157145 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.157471 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.157474 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.157476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.157561 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.157950 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.157956 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.158071 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.158189 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.158400 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":847,"spinlock_wait_cycles":2944,"thread_start_us":146,"threads_started":1}
I20260806 18:45:09.159701 11719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.124@KRBTEST.COM for oryx/127.1.245.125@KRBTEST.COM
I20260806 18:45:09.167550 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.160146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.0.0.1:58936)
0806 18:45:09.160259 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.160273 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.160333 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.160790 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.160793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.160925 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.161019 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.161024 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.161877 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.161879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.162494 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.162503 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.162892 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.162894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.162961 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.163352 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.163363 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.164531 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.166121 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.166123 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.166125 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.166431 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.166687 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.166689 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.166691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.166769 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.167122 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.167124 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.167207 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.167331 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.167432 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 18:45:09.167624 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.160306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58936 (local address 127.1.245.125:35987)
0806 18:45:09.160566 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.160570 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.160581 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:45:09.160616 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.160619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.160657 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.160726 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.161115 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.161779 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.162579 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.162802 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.162890 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.164672 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.164675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.164677 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.164701 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.166020 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.166526 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.166529 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.166530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.166613 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.166850 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.166852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.166853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.167043 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.167121 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.167383 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.167482 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":93,"threads_started":1}
I20260806 18:45:09.168337 11719 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.124@KRBTEST.COM for oryx/127.1.245.124@KRBTEST.COM
I20260806 18:45:09.175782 11723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.168831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:45209 (local address 127.0.0.1:49928)
0806 18:45:09.168951 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.168965 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.169048 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.169210 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.169212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.169360 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.169426 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.169430 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.170528 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.170531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.171063 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.171072 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.171498 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.171501 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.171563 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.172014 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.172025 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.173155 (+ 1130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.174653 (+ 1498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.174656 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.174657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.174842 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.175033 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.175035 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.175036 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.175097 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.175377 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.175379 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.175453 (+ 74us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.175523 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.175624 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:45:09.175802 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.168867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.1.245.124:45209)
0806 18:45:09.168972 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.168975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.169020 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:45:09.169104 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.169106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.169141 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.169203 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.169504 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.169994 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.171161 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.171388 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.171460 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.173301 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.173304 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.173306 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.173327 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.174549 (+ 1222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.174910 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.174912 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.174913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.174975 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.175152 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.175154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.175155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.175307 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.175375 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.175557 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.175679 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":39}
I20260806 18:45:09.178711 10968 heartbeater.cc:503] Master 127.1.245.126:37393 was elected leader, sending a full tablet report...
I20260806 18:45:09.178977 11719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Bootstrap starting.
I20260806 18:45:09.181499 11719 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:09.182257 11719 log.cc:824] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:09.184214 11719 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: No bootstrap required, opened a new log
I20260806 18:45:09.187311 11719 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } }
I20260806 18:45:09.187525 11719 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:09.187559 11719 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9304d77916a4a1bacb81ddad68f9845, State: Initialized, Role: FOLLOWER
I20260806 18:45:09.188010 11719 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } }
I20260806 18:45:09.188545 11726 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } } }
I20260806 18:45:09.188673 11726 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:45:09.189275 11719 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: configured and running, proceeding with master startup.
W20260806 18:45:09.194161 11747 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:45:09.194257 11747 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:45:09.197657 11499 master_runner.cc:428] Detected that this master b9304d77916a4a1bacb81ddad68f9845 is joining an existing cluster
I20260806 18:45:09.197723 11499 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.245.124:45209
I20260806 18:45:09.208910 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.200065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34584 (local address 127.1.245.126:37393)
0806 18:45:09.200182 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.200185 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.200403 (+ 218us) server_negotiation.cc:407] Connection header received
0806 18:45:09.200699 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.200702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.200753 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.200822 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.201357 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.202110 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.203318 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.203636 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.203714 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.205458 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.205463 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.205465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.205496 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.207049 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.207548 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.207552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.207554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.207643 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.207951 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.207954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.207956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.208164 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.208240 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.208609 (+ 369us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.208739 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:45:09.209446 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.199962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:34584)
0806 18:45:09.200328 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.200340 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.200599 (+ 259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.200850 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.200852 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.201081 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.201213 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.201219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.202222 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.202225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.203187 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.203195 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.203891 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.203894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.203986 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.204849 (+ 863us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.204858 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.205308 (+ 450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.207172 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.207175 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.207177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.207430 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.207757 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.207760 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.207762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.207847 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.208327 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.208330 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.208433 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.209142 (+ 709us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.209285 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":105,"threads_started":1}
I20260806 18:45:09.209440 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 10783
I20260806 18:45:09.214536 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.200501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58944 (local address 127.1.245.125:35987)
0806 18:45:09.200608 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.200612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.201284 (+ 672us) server_negotiation.cc:407] Connection header received
0806 18:45:09.201426 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.201429 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.201477 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.201546 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.202137 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.202961 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.204598 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.204939 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.205041 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.206293 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.206296 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.206299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.206349 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.207865 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.208366 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.208370 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.208372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.208458 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.208753 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.208756 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.208758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.208968 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.209036 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.209348 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.209472 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:45:09.214986 11758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.200431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.0.0.1:58944)
0806 18:45:09.201176 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.201201 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.201300 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.201656 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.201659 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.201884 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.201993 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.202000 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.203097 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.203100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.204486 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.204495 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.205069 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.205072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.205156 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.205712 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.205771 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.206175 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.207966 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.207969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.207971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.208248 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.208562 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.208566 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.208568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.208659 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.209104 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.209106 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.209189 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.214669 (+ 5480us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.214833 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":21,"thread_start_us":86,"threads_started":1}
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041907, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.126@KRBTEST.COM for oryx/127.1.245.124@KRBTEST.COM
I20260806 18:45:09.234536 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:41519
--local_ip_for_outbound_sockets=127.1.245.65
--tserver_master_addrs=127.1.245.126:37393
--webserver_port=35283
--webserver_interface=127.1.245.65
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab
--principal=oryx/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987,127.1.245.124:45209 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:45:09.234872 11722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.201077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49944 (local address 127.1.245.124:45209)
0806 18:45:09.201176 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.201180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.201599 (+ 419us) server_negotiation.cc:407] Connection header received
0806 18:45:09.201727 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.201730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.201764 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.201806 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.202487 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.203221 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.204415 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.204690 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.204754 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.231356 (+ 26602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.231360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.231363 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.231385 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.233219 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.233645 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.233648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.233650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.233722 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.233970 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.233973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.233974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.234175 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.234241 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.234582 (+ 341us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.234700 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 18:45:09.235530 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.212795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:45209 (local address 127.0.0.1:49960)
0806 18:45:09.215230 (+ 2435us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.215246 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.215330 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.227438 (+ 12108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.227441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.227641 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.227751 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.227759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.228547 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.228551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.229097 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.229107 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.229623 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.229627 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.229721 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.230216 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.230228 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.231787 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.233934 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.233937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.233939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.234166 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.234525 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.234528 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.234530 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.234616 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.235036 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.235040 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.235150 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.235226 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.235373 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2348}
I20260806 18:45:09.236125 11757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.200761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:45209 (local address 127.0.0.1:49944)
0806 18:45:09.201533 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.201544 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.201646 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.201945 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.201948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.202222 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.202371 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.202380 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.203340 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.203343 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.204277 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.204289 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.205163 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.205167 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.205299 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.230760 (+ 25461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.230776 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.231201 (+ 425us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.233309 (+ 2108us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.233311 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.233314 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.233545 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.233814 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.233816 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.233818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.233892 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.234308 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.234332 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.234418 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.235780 (+ 1362us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.235923 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":711,"mutex_wait_us":75,"thread_start_us":105,"threads_started":1}
I20260806 18:45:09.236135 11760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.213024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49960 (local address 127.1.245.124:45209)
0806 18:45:09.227107 (+ 14083us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.227114 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.227129 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:45:09.227210 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.227212 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.227274 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.227369 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.227880 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.228432 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.229210 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.229499 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.229592 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.232012 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.232016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.232018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.232044 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.233820 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.234286 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.234288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.234290 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.234414 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.234737 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.234739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.234741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.234926 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.235803 (+ 877us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.235889 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.235981 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14000,"thread_start_us":1320,"threads_started":1}
I20260806 18:45:09.237373 10693 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.245.124:45209: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845"
member_type: NON_VOTER
last_known_addr {
host: "127.1.245.124"
port: 45209
}
attrs {
promote: true
}
}
dest_uuid: "246273a6f7bc49578d7729671bee2816"
cas_config_opid_index: 5
I20260806 18:45:09.237984 10693 consensus_queue.cc:237] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } }
I20260806 18:45:09.239261 11490 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b9304d77916a4a1bacb81ddad68f9845 (127.1.245.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.239908 11680 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:09.241063 11490 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.245.124:45209
I20260806 18:45:09.241279 11490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.241395 11490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:09.241997 11680 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Refusing update from remote peer 246273a6f7bc49578d7729671bee2816: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:45:09.242190 11499 master.cc:537] Master@127.1.245.124:45209 shutting down...
I20260806 18:45:09.246742 11493 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:09.247733 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.239421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.0.0.1:58956)
0806 18:45:09.242778 (+ 3357us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.242796 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.242895 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.243401 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.243404 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.243425 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:09.243525 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.243533 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.244600 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.244604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.246043 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.246057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.247261 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.247265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.247364 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.247421 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.247505 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.247591 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3276}
I20260806 18:45:09.248111 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.241593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58956 (local address 127.1.245.125:35987)
0806 18:45:09.241809 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.241815 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.242994 (+ 1179us) server_negotiation.cc:407] Connection header received
0806 18:45:09.243062 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.243065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.243133 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.243247 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:09.243668 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.244448 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.246210 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.246874 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.246988 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.247177 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.247882 (+ 705us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.247974 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260806 18:45:09.248684 11553 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 LEARNER]: Refusing update from remote peer 246273a6f7bc49578d7729671bee2816: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 18:45:09.249456 11493 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 18:45:09.251430 11553 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER b9304d77916a4a1bacb81ddad68f9845 (127.1.245.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.253634 11597 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 246273a6f7bc49578d7729671bee2816. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.253755 11597 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: This master's current role is: LEARNER
I20260806 18:45:09.253768 11493 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: attempting to promote NON_VOTER c77cb7e640c048a0bc093a588c5ed78d to VOTER
I20260806 18:45:09.253818 11499 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:09.253842 11597 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: NON_VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: true } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.253906 11597 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: This master's current role is: LEARNER
I20260806 18:45:09.254104 11499 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:09.254235 11499 tablet_replica.cc:330] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: stopping tablet replica
I20260806 18:45:09.254225 11493 consensus_queue.cc:237] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } }
I20260806 18:45:09.255890 11553 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 LEARNER]: Refusing update from remote peer 246273a6f7bc49578d7729671bee2816: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260806 18:45:09.256346 11490 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260806 18:45:09.258188 10671 consensus_peers.cc:487] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 -> Peer b9304d77916a4a1bacb81ddad68f9845 (127.1.245.124:45209): Couldn't send request to peer b9304d77916a4a1bacb81ddad68f9845. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 18:45:09.259943 11490 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c77cb7e640c048a0bc093a588c5ed78d (127.1.245.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.261691 11560 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, c77cb7e640c048a0bc093a588c5ed78d (127.1.245.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.263123 11766 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.263339 11766 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:45:09.263885 11493 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.264006 11493 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:09.264209 11490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.264297 11490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:09.271420 11499 master.cc:552] Master@127.1.245.124:45209 shutdown complete.
I20260806 18:45:09.271473 11499 master_runner.cc:305] Clearing existing system tablet
I20260806 18:45:09.271687 11499 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:09.271739 11499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:09.272802 11499 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:09.273526 11769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.273736 11499 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:09.273859 11499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:09.273897 11499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
uuid: "b9304d77916a4a1bacb81ddad68f9845"
format_stamp: "Formatted at 2026-08-06 18:45:09 on dist-test-slave-fgb3"
server_key: "7daab170dbc2e72930eda8b2dd50dcf3"
server_key_iv: "38f8dd300f89f9ca917f18c931cee33d"
server_key_version: "encryptionkey@0"
I20260806 18:45:09.275601 11499 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:09.277431 11499 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 18:45:09.277522 11499 log.cc:1196] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 18:45:09.277763 11499 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Deleting consensus metadata
I20260806 18:45:09.277904 11499 master_runner.cc:315] Copying system tablet from 127.1.245.126:37393
I20260806 18:45:09.279019 11499 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: tablet copy: Beginning tablet copy session from remote peer at address 127.1.245.126:37393
I20260806 18:45:09.286227 11781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.279548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:34586)
0806 18:45:09.279870 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.279882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.279994 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.280218 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.280221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.280414 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.280540 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.280548 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.281300 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.281303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.281894 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.281902 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.282368 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.282372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.282462 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.282940 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.282956 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.283264 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.284806 (+ 1542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.284809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.284812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.285002 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.285252 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.285255 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.285257 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.285337 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.285682 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.285685 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.285786 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.285870 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.286015 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":81,"threads_started":1}
I20260806 18:45:09.286260 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.279619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34586 (local address 127.1.245.126:37393)
0806 18:45:09.279757 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.279762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.279959 (+ 197us) server_negotiation.cc:407] Connection header received
0806 18:45:09.280074 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.280077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.280124 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.280193 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.280661 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.281196 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.282005 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.282228 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.282350 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.283401 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.283405 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.283408 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.283435 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.284704 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.285088 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.285092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.285094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.285170 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.285415 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.285418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.285420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.285593 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.285678 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.285937 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.286049 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:45:09.286994 10722 tablet_copy_service.cc:140] P 246273a6f7bc49578d7729671bee2816: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b9304d77916a4a1bacb81ddad68f9845 ({username='oryx', principal='oryx/127.1.245.124@KRBTEST.COM'} at 127.0.0.1:34586)
I20260806 18:45:09.287083 10722 tablet_copy_service.cc:161] P 246273a6f7bc49578d7729671bee2816: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b9304d77916a4a1bacb81ddad68f9845 at {username='oryx', principal='oryx/127.1.245.124@KRBTEST.COM'} at 127.0.0.1:34586: session id = b9304d77916a4a1bacb81ddad68f9845-00000000000000000000000000000000
I20260806 18:45:09.287969 10722 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 18:45:09.289115 11499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 18:45:09.291211 11499 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: tablet copy: Starting download of 0 data blocks...
I20260806 18:45:09.291396 11499 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: tablet copy: Starting download of 1 WAL segments...
I20260806 18:45:09.292743 11499 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 18:45:09.294370 10722 tablet_copy_service.cc:342] P 246273a6f7bc49578d7729671bee2816: Request end of tablet copy session b9304d77916a4a1bacb81ddad68f9845-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.245.124@KRBTEST.COM'} at 127.0.0.1:34586
I20260806 18:45:09.294450 10722 tablet_copy_service.cc:433] P 246273a6f7bc49578d7729671bee2816: ending tablet copy session b9304d77916a4a1bacb81ddad68f9845-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b9304d77916a4a1bacb81ddad68f9845
I20260806 18:45:09.302502 11499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:09.304126 11782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:09.304281 11786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.304490 11499 server_base.cc:1034] running on GCE node
W20260806 18:45:09.310497 11783 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.310858 11499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:09.311269 11499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:09.312481 11499 hybrid_clock.cc:648] HybridClock initialized: now 1786041909312467 us; error 35 us; skew 500 ppm
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:09.314510 11499 init.cc:377] Logged in from keytab as oryx/127.1.245.124@KRBTEST.COM (short username oryx)
I20260806 18:45:09.315049 11499 webserver.cc:460] Webserver started at http://127.1.245.124:38185/ using document root <none> and password file <none>
I20260806 18:45:09.315238 11499 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:09.315286 11499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:09.317168 11499 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 18:45:09.319386 11794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.319775 11499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 18:45:09.319940 11499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
uuid: "b9304d77916a4a1bacb81ddad68f9845"
format_stamp: "Formatted at 2026-08-06 18:45:09 on dist-test-slave-fgb3"
server_key: "7daab170dbc2e72930eda8b2dd50dcf3"
server_key_iv: "38f8dd300f89f9ca917f18c931cee33d"
server_key_version: "encryptionkey@0"
I20260806 18:45:09.320061 11499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:09.342020 11499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:09.344015 11499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:09.348881 11499 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.124:45209
I20260806 18:45:09.349493 11499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
I20260806 18:45:09.351073 11863 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:45:09.351886 11863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Bootstrap starting.
I20260806 18:45:09.354602 11862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.124:45209 every 8 connection(s)
I20260806 18:45:09.360682 11863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:45:09.360970 11863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Bootstrap complete.
I20260806 18:45:09.361483 11863 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } }
I20260806 18:45:09.361598 11863 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b9304d77916a4a1bacb81ddad68f9845, State: Initialized, Role: LEARNER
I20260806 18:45:09.361701 11863 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } }
I20260806 18:45:09.362036 11866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.362164 11863 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:09.362223 11866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:45:09.365694 11881 catalog_manager.cc:1259] Loaded cluster ID: 04423f5d736947abb745a66cce39e2f5
I20260806 18:45:09.365743 11881 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: loading cluster ID for follower catalog manager: success
I20260806 18:45:09.367417 11881 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: acquiring CA information for follower catalog manager: success
I20260806 18:45:09.368660 11881 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 18:45:09.397822 11763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:09.398226 11763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:09.398339 11763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:45:09.403878 11155 heartbeater.cc:503] Master 127.1.245.126:37393 was elected leader, sending a full tablet report...
W20260806 18:45:09.404469 11763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:09.404531 11763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:09.404616 11763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:09.412071 11763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab
--principal=oryx/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:41519
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=35283
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987,127.1.245.124:45209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:09.413290 11763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:09.414265 11763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:09.425016 11888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:09.425051 11890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:09.425028 11887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.425201 11763 server_base.cc:1034] running on GCE node
I20260806 18:45:09.425717 11763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:09.426451 11763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:09.427625 11763 hybrid_clock.cc:648] HybridClock initialized: now 1786041909427602 us; error 35 us; skew 500 ppm
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:09.430506 11763 init.cc:377] Logged in from keytab as oryx/127.1.245.65@KRBTEST.COM (short username oryx)
I20260806 18:45:09.431463 11763 webserver.cc:460] Webserver started at http://127.1.245.65:35283/ using document root <none> and password file <none>
I20260806 18:45:09.432047 11763 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:09.432137 11763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:09.436447 11763 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:45:09.438771 11903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.440088 11763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:45:09.440224 11763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "ea17dfec40434cd88bc6e868eb83fa32"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "1230ba912bf9e851bb990bbc18783751"
server_key_iv: "ed6931851151249ea645ccab855d73ea"
server_key_version: "encryptionkey@0"
I20260806 18:45:09.440810 11763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:09.459538 11763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:09.463076 11763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:09.463312 11763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:09.464013 11763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:09.464975 11763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:09.465024 11763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.465078 11763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:09.465106 11763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.474356 11763 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:41519
I20260806 18:45:09.474406 12051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:41519 every 8 connection(s)
I20260806 18:45:09.475425 11763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:09.485023 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 11763
I20260806 18:45:09.485229 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 10973
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.125@KRBTEST.COM
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.124@KRBTEST.COM
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:09.492098 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.477521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:59153 (local address 127.1.245.124:45209)
0806 18:45:09.477995 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.478009 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.478390 (+ 381us) server_negotiation.cc:407] Connection header received
0806 18:45:09.479533 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.479536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.479586 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.479642 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.480966 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.481661 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.483309 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.483605 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.483683 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.487545 (+ 3862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.487549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.487553 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.487581 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.489055 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.489616 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.489620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.489622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.489716 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.490113 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.490116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.490118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.490349 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.490445 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.491830 (+ 1385us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.491961 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":87,"threads_started":1}
I20260806 18:45:09.493376 12061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.477694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:45209 (local address 127.1.245.65:59153)
0806 18:45:09.478209 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.478264 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.479310 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.479836 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.479842 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.480329 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.480819 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.480828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.481810 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.481813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.483193 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.483204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.483721 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.483724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.483809 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.484274 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.484288 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.487106 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.489167 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.489173 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.489191 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.489516 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.489813 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.489817 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.489819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.489990 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.491175 (+ 1185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.491181 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.491512 (+ 331us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.491729 (+ 217us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.492082 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":196,"spinlock_wait_cycles":45440,"thread_start_us":174,"threads_started":1}
I20260806 18:45:09.494431 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.477694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.65:52299)
0806 18:45:09.478210 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.478264 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.479311 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.479834 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.479841 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.480299 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.480808 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.480822 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.482067 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.482070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.483193 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.483204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.484321 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.484324 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.484393 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.484780 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.484794 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.487337 (+ 2543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.489201 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.489204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.489206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.489542 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.489859 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.489862 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.489865 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.489979 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.493902 (+ 3923us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.493905 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.493995 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.494087 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.494211 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":42,"spinlock_wait_cycles":9856,"thread_start_us":153,"threads_started":1}
I20260806 18:45:09.494498 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.477247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:52299 (local address 127.1.245.126:37393)
0806 18:45:09.477402 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.477407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.478381 (+ 974us) server_negotiation.cc:407] Connection header received
0806 18:45:09.479514 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.479518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.479575 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.479646 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.480945 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.481768 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.483779 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.484094 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.484190 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.487523 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.487527 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.487530 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.487560 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.489080 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.489666 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.489669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.489671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.489756 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.490082 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.490085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.490087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.490347 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.490445 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.494174 (+ 3729us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.494347 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:45:09.494912 12052 heartbeater.cc:347] Connected to a master server at 127.1.245.124:45209
I20260806 18:45:09.495126 12062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.478776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.1.245.65:33531)
0806 18:45:09.479018 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.479032 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.479317 (+ 285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.479836 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.479842 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.480328 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.480822 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.480828 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.481873 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.481876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.482937 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.482947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.483402 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.483404 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.483541 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.484124 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.484143 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.486812 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.492941 (+ 6129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.492945 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.492947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.493137 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.493402 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.493405 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.493407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.493490 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.493962 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.493965 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.494066 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.494837 (+ 771us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.494974 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":229,"spinlock_wait_cycles":17920,"thread_start_us":86,"threads_started":1}
I20260806 18:45:09.495157 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.478922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:33531 (local address 127.1.245.125:35987)
0806 18:45:09.479045 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.479049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.479091 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:45:09.479533 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.479536 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.479586 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.479639 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.480974 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.481769 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.483049 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.483270 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.483374 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.490870 (+ 7496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.490874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.490877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.490907 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.492836 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.493233 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.493236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.493237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.493318 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.493603 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.493607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.493609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.493813 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.493888 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.494899 (+ 1011us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.495021 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:45:09.495164 12052 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.495811 12056 heartbeater.cc:347] Connected to a master server at 127.1.245.125:35987
I20260806 18:45:09.495896 12052 heartbeater.cc:511] Master 127.1.245.124:45209 requested a full tablet report, sending...
I20260806 18:45:09.495898 12056 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.496112 12056 heartbeater.cc:511] Master 127.1.245.125:35987 requested a full tablet report, sending...
I20260806 18:45:09.497592 11544 ts_manager.cc:194] Registered new tserver with Master: ea17dfec40434cd88bc6e868eb83fa32 (127.1.245.65:41519)
I20260806 18:45:09.497629 11804 ts_manager.cc:194] Registered new tserver with Master: ea17dfec40434cd88bc6e868eb83fa32 (127.1.245.65:41519)
I20260806 18:45:09.497743 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:37589
--local_ip_for_outbound_sockets=127.1.245.66
--tserver_master_addrs=127.1.245.126:37393
--webserver_port=35799
--webserver_interface=127.1.245.66
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab
--principal=oryx/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987,127.1.245.124:45209 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:45:09.499130 12053 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37393
I20260806 18:45:09.499222 12053 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.499375 12053 heartbeater.cc:511] Master 127.1.245.126:37393 requested a full tablet report, sending...
I20260806 18:45:09.500202 10693 ts_manager.cc:194] Re-registered known tserver with Master: ea17dfec40434cd88bc6e868eb83fa32 (127.1.245.65:41519)
I20260806 18:45:09.500787 10693 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:52299
W20260806 18:45:09.614741 12064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:09.614971 12064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:09.615018 12064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:09.619102 12064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:09.619156 12064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:09.619231 12064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:09.624375 12064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab
--principal=oryx/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:37589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=35799
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987,127.1.245.124:45209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:09.625414 12064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:09.626407 12064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:45:09.630364 11343 heartbeater.cc:503] Master 127.1.245.126:37393 was elected leader, sending a full tablet report...
W20260806 18:45:09.636011 12072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:09.636098 12071 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.636091 12064 server_base.cc:1034] running on GCE node
W20260806 18:45:09.636029 12074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.636600 12064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:09.637219 12064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:09.638412 12064 hybrid_clock.cc:648] HybridClock initialized: now 1786041909638385 us; error 35 us; skew 500 ppm
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:09.641065 12064 init.cc:377] Logged in from keytab as oryx/127.1.245.66@KRBTEST.COM (short username oryx)
I20260806 18:45:09.641929 12064 webserver.cc:460] Webserver started at http://127.1.245.66:35799/ using document root <none> and password file <none>
I20260806 18:45:09.642531 12064 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:09.642616 12064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:09.646584 12064 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:45:09.648667 12086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.649945 12064 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:09.650064 12064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "76a9cf82ac3e485eb75e421bd6565b59"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "acb7576fa11691726696f37625e9eabc"
server_key_iv: "4521883f28164701bbea65dc3a5683cf"
server_key_version: "encryptionkey@0"
I20260806 18:45:09.650626 12064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:09.684821 12064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:09.688119 12064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:09.688309 12064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:09.688939 12064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:09.689873 12064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:09.689924 12064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.689980 12064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:09.690006 12064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.698855 12064 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:37589
I20260806 18:45:09.698912 12253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:37589 every 8 connection(s)
I20260806 18:45:09.699995 12064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:09.704787 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.700575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49966 (local address 127.1.245.124:45209)
0806 18:45:09.700865 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.700870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.700906 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:45:09.700961 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.700964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.701031 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.701132 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:09.701438 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.702201 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.703642 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.704262 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.704366 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.704523 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.704590 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.704660 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260806 18:45:09.704787 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.700482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:45209 (local address 127.0.0.1:49966)
0806 18:45:09.700726 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.700744 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.700848 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.701144 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.701147 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.701168 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:09.701274 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.701281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.702360 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.702363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.703521 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.703536 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.704382 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.704386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.704475 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.704497 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.704572 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.704650 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260806 18:45:09.705559 11820 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 18:45:09.706149 11866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 246273a6f7bc49578d7729671bee2816. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.706303 11866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: This master's current role is: LEARNER
I20260806 18:45:09.706300 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 12064
I20260806 18:45:09.706472 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 11165
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.124@KRBTEST.COM
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.125@KRBTEST.COM
I20260806 18:45:09.721439 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.703865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:50031 (local address 127.1.245.125:35987)
0806 18:45:09.703987 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.703991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.704111 (+ 120us) server_negotiation.cc:407] Connection header received
0806 18:45:09.704930 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.704933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.704979 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.705053 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.709916 (+ 4863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.710683 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.713576 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.713890 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.713951 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.717080 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.717084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.717086 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.717116 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.718980 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719680 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.719683 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.719685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.719768 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.720141 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.720145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.720146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.720367 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.720431 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.721147 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.721278 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:45:09.721442 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.702455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:50711 (local address 127.1.245.124:45209)
0806 18:45:09.702772 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.702776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.704078 (+ 1302us) server_negotiation.cc:407] Connection header received
0806 18:45:09.704699 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.704703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.704756 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.704814 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.708875 (+ 4061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.709657 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.712635 (+ 2978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.712948 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.713025 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.717267 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.717271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.717274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.717303 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.718927 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719681 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.719684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.719686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.719780 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.720144 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.720147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.720149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.720357 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.720434 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.721147 (+ 713us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.721271 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":83,"threads_started":1}
I20260806 18:45:09.721470 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.701810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:34881 (local address 127.1.245.126:37393)
0806 18:45:09.702786 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.702790 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.703347 (+ 557us) server_negotiation.cc:407] Connection header received
0806 18:45:09.704866 (+ 1519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.704869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.704916 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.704961 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.708924 (+ 3963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.709700 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.711338 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.711669 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.711737 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.716942 (+ 5205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.716946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.716949 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.716981 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.719080 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719669 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.719672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.719674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.719759 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.720120 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.720123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.720125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.720350 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.720429 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.721180 (+ 751us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.721305 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":909}
I20260806 18:45:09.722402 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:34405
--local_ip_for_outbound_sockets=127.1.245.67
--tserver_master_addrs=127.1.245.126:37393
--webserver_port=34517
--webserver_interface=127.1.245.67
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab
--principal=oryx/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987,127.1.245.124:45209 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:45:09.722951 12266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.702251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.66:34881)
0806 18:45:09.703212 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.703261 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.704473 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.706413 (+ 1940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.706418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.706583 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.708795 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.708802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.709841 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.709844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.711212 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.711222 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.711943 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.711947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.712149 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.712786 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.712806 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.716785 (+ 3979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.719197 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.719202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.719561 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.719857 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719860 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.719862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.720023 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.720496 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.720502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.720860 (+ 358us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.721657 (+ 797us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.722049 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"mutex_wait_us":2112,"spinlock_wait_cycles":18944,"thread_start_us":634,"threads_started":1}
I20260806 18:45:09.723482 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.703743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.1.245.66:50031)
0806 18:45:09.704033 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.704045 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.704652 (+ 607us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.705261 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.705270 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.705684 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.706351 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.706373 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.711095 (+ 4722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.711098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.711864 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.711873 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.714093 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.714096 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.714159 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.715087 (+ 928us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.715101 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.716914 (+ 1813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.719094 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.719120 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.719568 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.719880 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.719885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.720038 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.720496 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.720503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.720861 (+ 358us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.722077 (+ 1216us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.722205 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":73,"spinlock_wait_cycles":29952,"thread_start_us":80,"threads_started":1}
I20260806 18:45:09.724118 12261 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37393
I20260806 18:45:09.724282 12262 heartbeater.cc:347] Connected to a master server at 127.1.245.125:35987
I20260806 18:45:09.724356 12261 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.724365 12262 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.724792 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.702252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:45209 (local address 127.1.245.66:50711)
0806 18:45:09.703996 (+ 1744us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.704010 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.704481 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.705458 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.705460 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.705732 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.708725 (+ 2993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.708737 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.709838 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.709841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.712500 (+ 2659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.712510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.713177 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.713181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.713272 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.714463 (+ 1191us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.714480 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.716578 (+ 2098us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.719094 (+ 2516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719101 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.719120 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.719568 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.719880 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.719883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.719885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.720038 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.720496 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.720503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.720861 (+ 358us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.721656 (+ 795us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.724055 (+ 2399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1657,"mutex_wait_us":2845,"spinlock_wait_cycles":66688,"thread_start_us":127,"threads_started":1}
I20260806 18:45:09.724870 12262 heartbeater.cc:511] Master 127.1.245.125:35987 requested a full tablet report, sending...
I20260806 18:45:09.725332 12256 heartbeater.cc:347] Connected to a master server at 127.1.245.124:45209
I20260806 18:45:09.725404 12256 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.725549 12256 heartbeater.cc:511] Master 127.1.245.124:45209 requested a full tablet report, sending...
I20260806 18:45:09.725711 12261 heartbeater.cc:511] Master 127.1.245.126:37393 requested a full tablet report, sending...
I20260806 18:45:09.726078 11544 ts_manager.cc:194] Registered new tserver with Master: 76a9cf82ac3e485eb75e421bd6565b59 (127.1.245.66:37589)
I20260806 18:45:09.726290 11804 ts_manager.cc:194] Registered new tserver with Master: 76a9cf82ac3e485eb75e421bd6565b59 (127.1.245.66:37589)
I20260806 18:45:09.726339 10693 ts_manager.cc:194] Re-registered known tserver with Master: 76a9cf82ac3e485eb75e421bd6565b59 (127.1.245.66:37589)
I20260806 18:45:09.727046 10693 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:34881
W20260806 18:45:09.839118 12269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:09.839391 12269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:09.839435 12269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:09.843673 12269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:09.843741 12269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:09.843806 12269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:09.849280 12269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab
--principal=oryx/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:34405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=34517
--tserver_master_addrs=127.1.245.126:37393,127.1.245.125:35987,127.1.245.124:45209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:09.850421 12269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:09.851464 12269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:09.861135 12280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.861183 12269 server_base.cc:1034] running on GCE node
W20260806 18:45:09.861191 12277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:09.861433 12276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:09.861879 12269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:09.862584 12269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:09.863744 12269 hybrid_clock.cc:648] HybridClock initialized: now 1786041909863714 us; error 42 us; skew 500 ppm
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:09.866430 12269 init.cc:377] Logged in from keytab as oryx/127.1.245.67@KRBTEST.COM (short username oryx)
I20260806 18:45:09.867323 12269 webserver.cc:460] Webserver started at http://127.1.245.67:34517/ using document root <none> and password file <none>
I20260806 18:45:09.867918 12269 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:09.867996 12269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:09.871942 12269 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 18:45:09.874071 12296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.875350 12269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:45:09.875455 12269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "3fdb241e0e474cb3b1534857c34eaec7"
format_stamp: "Formatted at 2026-08-06 18:45:08 on dist-test-slave-fgb3"
server_key: "fb1a4b25ad794683de0d4e2e1098605a"
server_key_iv: "6bbc0dac1e89a696c445199dc4397dc4"
server_key_version: "encryptionkey@0"
I20260806 18:45:09.876003 12269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:09.892843 12269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:09.896070 12269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:09.896245 12269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:09.896899 12269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:09.897810 12269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:09.897857 12269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.897897 12269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:09.897924 12269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:09.907379 12269 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:34405
I20260806 18:45:09.907411 12453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:34405 every 8 connection(s)
I20260806 18:45:09.908390 12269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:09.910934 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 12269
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.124@KRBTEST.COM
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
Aug 06 18:45:09 dist-test-slave-fgb3 krb5kdc[10631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041909, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.125@KRBTEST.COM
I20260806 18:45:09.927489 11724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.911808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:33549 (local address 127.1.245.125:35987)
0806 18:45:09.911940 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.911945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.912640 (+ 695us) server_negotiation.cc:407] Connection header received
0806 18:45:09.913114 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.913117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.913176 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.913231 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.915245 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.915991 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.919585 (+ 3594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.919823 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.919898 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.923120 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.923124 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.923126 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.923149 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.924876 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.925598 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.925601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.925603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.925683 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.926085 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.926089 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.926090 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.926302 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.926414 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.927163 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.927296 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:45:09.927495 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.913624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58960 (local address 127.1.245.125:35987)
0806 18:45:09.914589 (+ 965us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.914592 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.914605 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:09.914640 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.914642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.914680 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.915644 (+ 964us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.915748 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.916440 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.918634 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.919051 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.919540 (+ 489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.922727 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.922731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.922734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.922761 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.925369 (+ 2608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.925822 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.925825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.925826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.925908 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.926202 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.926205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.926207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.926408 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.927144 (+ 736us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.927204 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.927356 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":896,"spinlock_wait_cycles":4480,"thread_start_us":104,"threads_started":1}
I20260806 18:45:09.928617 12461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.910689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:45209 (local address 127.1.245.67:51071)
0806 18:45:09.912179 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.912191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.912799 (+ 608us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.913576 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.913585 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.914261 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.915075 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.915096 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.916233 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.916237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.917927 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.917937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.918912 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.918919 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.919159 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.919810 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.919833 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.921825 (+ 1992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.925084 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.925087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.925089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.925505 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.925833 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.925836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.925838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.925987 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.926475 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.926483 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.926860 (+ 377us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.927102 (+ 242us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.927463 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1413,"mutex_wait_us":348,"spinlock_wait_cycles":126720,"thread_start_us":151,"threads_started":1}
I20260806 18:45:09.928846 12458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.911661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.1.245.67:33549)
0806 18:45:09.912551 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.912562 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.912974 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.913575 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.913584 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.914265 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.915089 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.915102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.916150 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.916153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.919280 (+ 3127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.919291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.920067 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.920071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.920161 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.921523 (+ 1362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.921539 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.922980 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.925004 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.925010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.925029 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.925492 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.925788 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.925791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.925792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.925985 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.926478 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.926485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.926860 (+ 375us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.927680 (+ 820us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.927826 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":1610,"spinlock_wait_cycles":18176,"thread_start_us":90,"threads_started":1}
I20260806 18:45:09.929518 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.910424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:51071 (local address 127.1.245.124:45209)
0806 18:45:09.910550 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.910554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.912292 (+ 1738us) server_negotiation.cc:407] Connection header received
0806 18:45:09.913134 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.913138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.913205 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.913293 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.915261 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.916089 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.918106 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.918540 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.918650 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.922022 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.922026 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.922029 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.922111 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.924964 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.925617 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.925620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.925623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.925711 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.926092 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.926096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.926097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.926305 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.926417 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.929235 (+ 2818us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.929371 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:45:09.929967 12456 heartbeater.cc:347] Connected to a master server at 127.1.245.125:35987
I20260806 18:45:09.930084 12454 heartbeater.cc:347] Connected to a master server at 127.1.245.124:45209
I20260806 18:45:09.930227 12456 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.930269 10751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.910246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:37617 (local address 127.1.245.126:37393)
0806 18:45:09.910431 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.910436 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.911899 (+ 1463us) server_negotiation.cc:407] Connection header received
0806 18:45:09.913868 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.913872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.913935 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.914030 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.915265 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.916089 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.918466 (+ 2377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.918773 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.918848 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.923102 (+ 4254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.923106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.923108 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.923136 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.924964 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.927206 (+ 2242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.927209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.927211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.927305 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.928641 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.928645 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.928647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.928861 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.928942 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.930006 (+ 1064us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.930122 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260806 18:45:09.930824 12456 heartbeater.cc:511] Master 127.1.245.125:35987 requested a full tablet report, sending...
I20260806 18:45:09.932011 11544 ts_manager.cc:194] Registered new tserver with Master: 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405)
I20260806 18:45:09.930227 12454 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.932662 12454 heartbeater.cc:511] Master 127.1.245.124:45209 requested a full tablet report, sending...
I20260806 18:45:09.932835 12469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.912908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.1.245.126:37393)
0806 18:45:09.917793 (+ 4885us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.917809 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.917826 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:45:09.917869 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.917872 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.917922 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.920420 (+ 2498us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.920539 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.921335 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.922531 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.922947 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.923042 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.925105 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.925108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.925111 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.925132 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.927084 (+ 1952us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.930691 (+ 3607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.930693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.930694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.930761 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.931144 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.931147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.931149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.931361 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.931484 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.932190 (+ 706us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.932623 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3589,"thread_start_us":94,"threads_started":1}
I20260806 18:45:09.933101 12460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.910690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.67:37617)
0806 18:45:09.911731 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.911790 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.912787 (+ 997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.914161 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.914164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.914397 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:09.915078 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.915096 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.916246 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.916249 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.918238 (+ 1989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.918349 (+ 111us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.918990 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.918993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.919118 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.920587 (+ 1469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:09.920598 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:09.922561 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:09.926014 (+ 3453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.926017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.926020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.926229 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.927413 (+ 1184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:09.927416 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:09.927418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:09.927498 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:09.929382 (+ 1884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:09.929385 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:09.929472 (+ 87us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.929792 (+ 320us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.929931 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":851,"mutex_wait_us":253,"spinlock_wait_cycles":3456,"thread_start_us":155,"threads_started":1}
I20260806 18:45:09.933499 11804 ts_manager.cc:194] Registered new tserver with Master: 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405)
I20260806 18:45:09.939479 12455 heartbeater.cc:347] Connected to a master server at 127.1.245.126:37393
I20260806 18:45:09.939594 12455 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:09.939769 12455 heartbeater.cc:511] Master 127.1.245.126:37393 requested a full tablet report, sending...
I20260806 18:45:09.940426 10694 ts_manager.cc:194] Re-registered known tserver with Master: 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405)
I20260806 18:45:09.941143 10694 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:37617
I20260806 18:45:09.941926 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.913666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49970 (local address 127.1.245.124:45209)
0806 18:45:09.913774 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.913778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.923978 (+ 10200us) server_negotiation.cc:407] Connection header received
0806 18:45:09.924092 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.924095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.924150 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.924225 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:09.928589 (+ 4364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.929305 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.931251 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.931644 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.931753 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.937772 (+ 6019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:09.937776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:09.937779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:09.937808 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:09.940090 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.940590 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.940593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.940595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.940716 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:09.941046 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:09.941050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:09.941052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:09.941278 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:09.941410 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.941662 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.941802 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:45:09.943195 10694 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34598:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:45:09.947118 10694 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:09.950284 12475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: NON_VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: true } } }
I20260806 18:45:09.950443 12475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:09.953508 12475 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: attempting to promote NON_VOTER b9304d77916a4a1bacb81ddad68f9845 to VOTER
I20260806 18:45:09.953847 12475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } }
I20260806 18:45:09.955106 11820 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 LEARNER]: Refusing update from remote peer 246273a6f7bc49578d7729671bee2816: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:45:09.955168 11560 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Refusing update from remote peer 246273a6f7bc49578d7729671bee2816: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 18:45:09.955443 12477 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 18:45:09.955590 12476 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 18:45:09.957185 12477 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b9304d77916a4a1bacb81ddad68f9845 (127.1.245.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:09.957607 11560 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b9304d77916a4a1bacb81ddad68f9845 (127.1.245.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:09.957839 11820 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, b9304d77916a4a1bacb81ddad68f9845 (127.1.245.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:09.958812 12477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:09.958921 12477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:09.959054 12477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:09.959144 12477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:09.959283 11490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:09.959380 11490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:09.959641 11866 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:09.959743 11866 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:45:09.961728 12487 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "246273a6f7bc49578d7729671bee2816" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:09.961829 12487 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:45:09.970825 11487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.965424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:37589 (local address 127.0.0.1:35544)
0806 18:45:09.965581 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.965596 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.965676 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.966498 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.966501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.966521 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:09.966631 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.966638 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.968179 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.968183 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.969341 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.969351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.970418 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.970422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.970520 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.970546 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.970619 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.970683 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":3}
I20260806 18:45:09.970923 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.965565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35544 (local address 127.1.245.66:37589)
0806 18:45:09.965930 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.965943 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.965969 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:45:09.966054 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.966060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.966296 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.966819 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:09.966977 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.968083 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.969441 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.970258 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.970416 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.970568 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.970659 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.970761 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":136,"threads_started":1}
I20260806 18:45:09.972189 12489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.965916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:41519 (local address 127.0.0.1:45596)
0806 18:45:09.966689 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.966700 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.966808 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.967367 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.967375 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.967394 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:09.967483 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.967490 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.969019 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.969022 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.970031 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.970040 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.971725 (+ 1685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.971729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.971911 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.971931 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.971985 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.972057 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"mutex_wait_us":47,"thread_start_us":106,"threads_started":1}
I20260806 18:45:09.972229 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.966342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.1.245.65:41519)
0806 18:45:09.967002 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.967014 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.967031 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:45:09.967087 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.967091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.967219 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.967656 (+ 437us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:09.967836 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.968902 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.970176 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.970966 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.971830 (+ 864us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.971966 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.972015 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.972098 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":63,"threads_started":1}
I20260806 18:45:09.972262 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.966138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.1.245.67:34405)
0806 18:45:09.966570 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:09.966581 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:09.967712 (+ 1131us) server_negotiation.cc:407] Connection header received
0806 18:45:09.967901 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:09.967906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:09.968077 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:09.968194 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:09.968539 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.969309 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.970687 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.971332 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.971397 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.971502 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:09.972042 (+ 540us) server_negotiation.cc:299] Negotiation successful
0806 18:45:09.972137 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":119,"threads_started":1}
I20260806 18:45:09.972735 12490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:09.965987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:34405 (local address 127.0.0.1:34762)
0806 18:45:09.967611 (+ 1624us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:09.967625 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:09.967776 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:09.968337 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:09.968339 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:09.968354 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:09.968432 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.968436 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.969418 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.969422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.970549 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:09.970561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:09.971716 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:09.971720 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:09.971880 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:09.971903 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:45:09.972537 (+ 634us) client_negotiation.cc:240] Negotiation successful
0806 18:45:09.972612 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1563,"mutex_wait_us":36,"thread_start_us":110,"threads_started":1}
I20260806 18:45:09.973867 12154 tablet_service.cc:1467] Processing CreateTablet for tablet 9236193d8e2b466cb2703bedbf5054fc (DEFAULT_TABLE table=test-table [id=f3d09799943e45b3a6717c0575a39a53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:09.974857 12154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9236193d8e2b466cb2703bedbf5054fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:09.975236 12363 tablet_service.cc:1467] Processing CreateTablet for tablet 9236193d8e2b466cb2703bedbf5054fc (DEFAULT_TABLE table=test-table [id=f3d09799943e45b3a6717c0575a39a53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:09.975236 11969 tablet_service.cc:1467] Processing CreateTablet for tablet 9236193d8e2b466cb2703bedbf5054fc (DEFAULT_TABLE table=test-table [id=f3d09799943e45b3a6717c0575a39a53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:09.976239 11969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9236193d8e2b466cb2703bedbf5054fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:09.976253 12363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9236193d8e2b466cb2703bedbf5054fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:09.981716 12497 tablet_bootstrap.cc:492] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Bootstrap starting.
I20260806 18:45:09.983984 12499 tablet_bootstrap.cc:492] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Bootstrap starting.
I20260806 18:45:09.984369 12497 tablet_bootstrap.cc:654] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:09.985339 12497 log.cc:824] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:09.986053 12499 tablet_bootstrap.cc:654] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:09.986673 12498 tablet_bootstrap.cc:492] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Bootstrap starting.
I20260806 18:45:09.986891 12499 log.cc:824] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:09.987541 12497 tablet_bootstrap.cc:492] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: No bootstrap required, opened a new log
I20260806 18:45:09.987743 12497 ts_tablet_manager.cc:1397] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:45:09.988651 12498 tablet_bootstrap.cc:654] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:09.989066 12499 tablet_bootstrap.cc:492] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: No bootstrap required, opened a new log
I20260806 18:45:09.989248 12499 ts_tablet_manager.cc:1397] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260806 18:45:09.989355 12498 log.cc:824] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:09.991075 12498 tablet_bootstrap.cc:492] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: No bootstrap required, opened a new log
I20260806 18:45:09.991256 12498 ts_tablet_manager.cc:1397] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:45:09.992350 12497 raft_consensus.cc:348] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:09.992658 12497 raft_consensus.cc:374] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:09.992714 12497 raft_consensus.cc:729] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76a9cf82ac3e485eb75e421bd6565b59, State: Initialized, Role: FOLLOWER
I20260806 18:45:09.993218 12497 consensus_queue.cc:260] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:09.993805 12499 raft_consensus.cc:348] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:09.994055 12499 raft_consensus.cc:374] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:09.994100 12499 raft_consensus.cc:729] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fdb241e0e474cb3b1534857c34eaec7, State: Initialized, Role: FOLLOWER
I20260806 18:45:09.994335 12497 ts_tablet_manager.cc:1428] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:45:09.994587 12499 consensus_queue.cc:260] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:09.995258 12498 raft_consensus.cc:348] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:09.995450 12498 raft_consensus.cc:374] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:09.995510 12498 raft_consensus.cc:729] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea17dfec40434cd88bc6e868eb83fa32, State: Initialized, Role: FOLLOWER
I20260806 18:45:09.995520 12499 ts_tablet_manager.cc:1428] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260806 18:45:09.995867 12455 heartbeater.cc:503] Master 127.1.245.126:37393 was elected leader, sending a full tablet report...
I20260806 18:45:09.995949 12498 consensus_queue.cc:260] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:09.997118 12261 heartbeater.cc:503] Master 127.1.245.126:37393 was elected leader, sending a full tablet report...
I20260806 18:45:09.997782 12498 ts_tablet_manager.cc:1428] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:45:09.998090 12053 heartbeater.cc:503] Master 127.1.245.126:37393 was elected leader, sending a full tablet report...
W20260806 18:45:10.159745 12457 tablet.cc:2369] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:45:10.203156 12263 tablet.cc:2369] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:45:10.222095 12507 raft_consensus.cc:482] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:10.222362 12507 raft_consensus.cc:504] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:10.223598 12507 leader_election.cc:288] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405), ea17dfec40434cd88bc6e868eb83fa32 (127.1.245.65:41519)
I20260806 18:45:10.227113 12267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.223908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:41519 (local address 127.1.245.66:60231)
0806 18:45:10.224100 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:10.224114 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:10.224207 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:10.224427 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:10.224429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:10.224444 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:10.224549 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:10.224554 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.225250 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.225253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:10.226029 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:10.226038 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.226684 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.226688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:10.226791 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.226836 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 18:45:10.226900 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:45:10.226974 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":4}
I20260806 18:45:10.227113 12495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.223999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:60231 (local address 127.1.245.65:41519)
0806 18:45:10.224106 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:10.224111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:10.224163 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:45:10.224304 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:10.224307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:10.224347 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:10.224409 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:10.224643 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.225150 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.226147 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.226577 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.226684 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.226740 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:10.226924 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 18:45:10.226990 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:45:10.227123 12494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.223957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:44653 (local address 127.1.245.67:34405)
0806 18:45:10.224083 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:10.224087 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:10.224103 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:45:10.224247 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:10.224249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:10.224292 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:10.224356 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:10.224600 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.225112 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.226168 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.226595 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.226701 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.226754 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:10.226939 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:45:10.227007 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:45:10.227149 12268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.223831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:34405 (local address 127.1.245.66:44653)
0806 18:45:10.224031 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:10.224047 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:10.224142 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:10.224372 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:10.224374 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:10.224388 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:10.224505 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:10.224510 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.225209 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.225213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:10.226053 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:10.226062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.226701 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.226705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:10.226807 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.226851 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 18:45:10.226913 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 18:45:10.226983 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":68}
W20260806 18:45:10.227636 12057 tablet.cc:2369] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:45:10.227802 11995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236193d8e2b466cb2703bedbf5054fc" candidate_uuid: "76a9cf82ac3e485eb75e421bd6565b59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea17dfec40434cd88bc6e868eb83fa32" is_pre_election: true
I20260806 18:45:10.227803 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236193d8e2b466cb2703bedbf5054fc" candidate_uuid: "76a9cf82ac3e485eb75e421bd6565b59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fdb241e0e474cb3b1534857c34eaec7" is_pre_election: true
I20260806 18:45:10.228121 11995 raft_consensus.cc:2454] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76a9cf82ac3e485eb75e421bd6565b59 in term 0.
I20260806 18:45:10.228121 12394 raft_consensus.cc:2454] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76a9cf82ac3e485eb75e421bd6565b59 in term 0.
I20260806 18:45:10.228605 12090 leader_election.cc:302] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76a9cf82ac3e485eb75e421bd6565b59, ea17dfec40434cd88bc6e868eb83fa32; no voters:
I20260806 18:45:10.228859 12507 raft_consensus.cc:2781] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:10.228960 12507 raft_consensus.cc:482] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:10.229012 12507 raft_consensus.cc:3037] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:10.230021 12507 raft_consensus.cc:504] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:10.230382 12507 leader_election.cc:288] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [CANDIDATE]: Term 1 election: Requested vote from peers 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405), ea17dfec40434cd88bc6e868eb83fa32 (127.1.245.65:41519)
I20260806 18:45:10.230731 11995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236193d8e2b466cb2703bedbf5054fc" candidate_uuid: "76a9cf82ac3e485eb75e421bd6565b59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea17dfec40434cd88bc6e868eb83fa32"
I20260806 18:45:10.230731 12394 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9236193d8e2b466cb2703bedbf5054fc" candidate_uuid: "76a9cf82ac3e485eb75e421bd6565b59" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fdb241e0e474cb3b1534857c34eaec7"
I20260806 18:45:10.230868 11995 raft_consensus.cc:3037] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:10.230868 12394 raft_consensus.cc:3037] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:10.232108 11995 raft_consensus.cc:2454] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76a9cf82ac3e485eb75e421bd6565b59 in term 1.
I20260806 18:45:10.232108 12394 raft_consensus.cc:2454] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76a9cf82ac3e485eb75e421bd6565b59 in term 1.
I20260806 18:45:10.232410 12088 leader_election.cc:302] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3fdb241e0e474cb3b1534857c34eaec7, 76a9cf82ac3e485eb75e421bd6565b59, ea17dfec40434cd88bc6e868eb83fa32; no voters:
I20260806 18:45:10.232580 12507 raft_consensus.cc:2781] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:10.232774 12507 raft_consensus.cc:686] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 LEADER]: Becoming Leader. State: Replica: 76a9cf82ac3e485eb75e421bd6565b59, State: Running, Role: LEADER
I20260806 18:45:10.233170 12507 consensus_queue.cc:237] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } }
I20260806 18:45:10.236732 10690 catalog_manager.cc:5495] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76a9cf82ac3e485eb75e421bd6565b59 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "76a9cf82ac3e485eb75e421bd6565b59" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } health_report { overall_health: UNKNOWN } } }
I20260806 18:45:10.273168 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.269891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.1.245.66:37589)
0806 18:45:10.270027 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:10.270031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:10.270045 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:10.270153 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:10.270156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:10.270211 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:10.270300 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:10.270563 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.271065 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.271887 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.272088 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.272155 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.272410 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:10.272839 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:10.272926 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:10.272967 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 18:45:10.273020 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:45:10.279683 11995 raft_consensus.cc:1261] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 1 FOLLOWER]: Refusing update from remote peer 76a9cf82ac3e485eb75e421bd6565b59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:10.279683 12394 raft_consensus.cc:1261] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 1 FOLLOWER]: Refusing update from remote peer 76a9cf82ac3e485eb75e421bd6565b59: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:10.280509 12507 consensus_queue.cc:1035] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:10.280717 12514 consensus_queue.cc:1035] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:10.293284 12521 mvcc.cc:205] Tried to move back new op lower bound from 7315627664497262592 to 7315627664317677568. Current Snapshot: MvccSnapshot[applied={T|T < 7315627664497262592}]
I20260806 18:45:10.298293 10690 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:10.298485 10690 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34598:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:10.302663 10690 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 246273a6f7bc49578d7729671bee2816: Sending DeleteTablet for 3 replicas of tablet 9236193d8e2b466cb2703bedbf5054fc
I20260806 18:45:10.303531 12154 tablet_service.cc:1514] Processing DeleteTablet for tablet 9236193d8e2b466cb2703bedbf5054fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:10 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35544
I20260806 18:45:10.304042 11969 tablet_service.cc:1514] Processing DeleteTablet for tablet 9236193d8e2b466cb2703bedbf5054fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:10 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:45596
I20260806 18:45:10.304380 12532 tablet_replica.cc:330] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: stopping tablet replica
I20260806 18:45:10.304705 12532 raft_consensus.cc:2229] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:10.304808 12363 tablet_service.cc:1514] Processing DeleteTablet for tablet 9236193d8e2b466cb2703bedbf5054fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:10 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:34762
I20260806 18:45:10.305058 12532 raft_consensus.cc:2258] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:10.305383 12536 tablet_replica.cc:330] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: stopping tablet replica
I20260806 18:45:10.305670 12536 raft_consensus.cc:2229] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:10.305989 12536 raft_consensus.cc:2258] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:10.306463 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 10649
I20260806 18:45:10.311614 12532 ts_tablet_manager.cc:1905] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:10.314728 12532 ts_tablet_manager.cc:1918] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:10.314845 12532 log.cc:1196] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/9236193d8e2b466cb2703bedbf5054fc
I20260806 18:45:10.315166 12532 ts_tablet_manager.cc:1939] T 9236193d8e2b466cb2703bedbf5054fc P 76a9cf82ac3e485eb75e421bd6565b59: Deleting consensus metadata
I20260806 18:45:10.316375 12535 tablet_replica.cc:330] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: stopping tablet replica
I20260806 18:45:10.316928 12535 raft_consensus.cc:2229] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:10.317407 12535 raft_consensus.cc:2258] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:10.321846 12535 ts_tablet_manager.cc:1905] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:10.325233 12535 ts_tablet_manager.cc:1918] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:10.325752 12536 ts_tablet_manager.cc:1905] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:10.328400 12536 ts_tablet_manager.cc:1918] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:10.328526 12536 log.cc:1196] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/9236193d8e2b466cb2703bedbf5054fc
I20260806 18:45:10.328873 12536 ts_tablet_manager.cc:1939] T 9236193d8e2b466cb2703bedbf5054fc P 3fdb241e0e474cb3b1534857c34eaec7: Deleting consensus metadata
I20260806 18:45:10.329617 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.304811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58962 (local address 127.1.245.125:35987)
0806 18:45:10.304921 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:10.304926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:10.306738 (+ 1812us) server_negotiation.cc:407] Connection header received
0806 18:45:10.311737 (+ 4999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:10.311743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:10.311813 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:10.311881 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:10.313235 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.314381 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.317795 (+ 3414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.318308 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.318442 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.325501 (+ 7059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:10.325506 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:10.325509 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:10.325547 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:10.327511 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:10.328059 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:10.328063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:10.328065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:10.328155 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:10.328470 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:10.328473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:10.328475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:10.328703 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:10.328805 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:10.329302 (+ 497us) server_negotiation.cc:299] Negotiation successful
0806 18:45:10.329447 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:45:10.330336 12535 log.cc:1196] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/9236193d8e2b466cb2703bedbf5054fc
I20260806 18:45:10.330732 12535 ts_tablet_manager.cc:1939] T 9236193d8e2b466cb2703bedbf5054fc P ea17dfec40434cd88bc6e868eb83fa32: Deleting consensus metadata
W20260806 18:45:10.332536 12299 connection.cc:535] server connection from 127.0.0.1:34762 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 18:45:10.332536 11904 connection.cc:535] server connection from 127.0.0.1:45596 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 18:45:10.332981 12089 connection.cc:535] server connection from 127.0.0.1:35544 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:45:10.342374 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.306344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49980 (local address 127.1.245.124:45209)
0806 18:45:10.306461 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:10.306465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:10.331464 (+ 24999us) server_negotiation.cc:407] Connection header received
0806 18:45:10.331646 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:10.331651 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:10.331703 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:10.331753 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:10.332713 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.333536 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.334722 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.335069 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.335143 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.336797 (+ 1654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:10.336801 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:10.336804 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:10.336834 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:10.338699 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:10.340491 (+ 1792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:10.340495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:10.340497 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:10.340585 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:10.340860 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:10.340864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:10.340865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:10.341503 (+ 638us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:10.341628 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:10.342075 (+ 447us) server_negotiation.cc:299] Negotiation successful
0806 18:45:10.342185 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:45:10.346230 12472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.336542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.1.245.125:35987)
0806 18:45:10.336760 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:10.336765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:10.336778 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:10.336815 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:10.336818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:10.336861 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:10.336930 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:10.338191 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.339053 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.340202 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.340557 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.340627 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.342195 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:10.342199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:10.342201 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:10.342228 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:10.344394 (+ 2166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:10.344958 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:10.344961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:10.344963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:10.345061 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:10.345381 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:10.345385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:10.345387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:10.345574 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:10.345663 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:10.345994 (+ 331us) server_negotiation.cc:299] Negotiation successful
0806 18:45:10.346098 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147}
I20260806 18:45:10.346223 12265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:10.336650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49992 (local address 127.1.245.124:45209)
0806 18:45:10.337077 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:10.337081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:10.337498 (+ 417us) server_negotiation.cc:407] Connection header received
0806 18:45:10.337612 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:10.337616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:10.337660 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:10.337739 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:10.338478 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.339134 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.340516 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:10.340813 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:10.340878 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:10.342303 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:10.342306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:10.342308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:10.342362 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:10.344377 (+ 2015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:10.344902 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:10.344905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:10.344907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:10.345000 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:10.345307 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:10.345311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:10.345312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:10.345519 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:10.345664 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:10.345926 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 18:45:10.346049 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355}
I20260806 18:45:11.296290 12546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.295691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.66:42927)
0806 18:45:11.296059 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.296167 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":109,"threads_started":1}
W20260806 18:45:11.296686 12261 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:37393 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
I20260806 18:45:11.296904 12547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.296353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.67:56901)
0806 18:45:11.296701 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.296810 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":94,"threads_started":1}
W20260806 18:45:11.297184 12455 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:37393 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
I20260806 18:45:11.299108 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.298654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.65:32907)
0806 18:45:11.298878 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.299027 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":62,"threads_started":1}
W20260806 18:45:11.299392 12053 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:37393 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
I20260806 18:45:11.802745 12549 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 246273a6f7bc49578d7729671bee2816)
I20260806 18:45:11.803012 12549 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } }
I20260806 18:45:11.803784 12550 raft_consensus.cc:482] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 246273a6f7bc49578d7729671bee2816)
I20260806 18:45:11.803969 12550 raft_consensus.cc:504] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } }
I20260806 18:45:11.804296 12549 leader_election.cc:288] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 246273a6f7bc49578d7729671bee2816 (127.1.245.126:37393), c77cb7e640c048a0bc093a588c5ed78d (127.1.245.125:35987)
I20260806 18:45:11.804935 12550 leader_election.cc:288] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 246273a6f7bc49578d7729671bee2816 (127.1.245.126:37393), b9304d77916a4a1bacb81ddad68f9845 (127.1.245.124:45209)
I20260806 18:45:11.804908 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.804437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:35070)
0806 18:45:11.804720 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.804804 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":62,"threads_started":1}
W20260806 18:45:11.805400 11796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
I20260806 18:45:11.805632 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.804998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:35072)
0806 18:45:11.805419 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.805542 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":54,"threads_started":1}
W20260806 18:45:11.805910 11524 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
I20260806 18:45:11.806455 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.806160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:35084)
0806 18:45:11.806305 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.806387 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260806 18:45:11.806600 11796 leader_election.cc:334] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 246273a6f7bc49578d7729671bee2816 (127.1.245.126:37393): Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
I20260806 18:45:11.806978 12555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.806706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:35090)
0806 18:45:11.806883 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.806927 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260806 18:45:11.807070 11524 leader_election.cc:334] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 246273a6f7bc49578d7729671bee2816 (127.1.245.126:37393): Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
I20260806 18:45:11.808754 12554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.804543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58982 (local address 127.1.245.125:35987)
0806 18:45:11.804805 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:11.804809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:11.805029 (+ 220us) server_negotiation.cc:407] Connection header received
0806 18:45:11.805081 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:11.805084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:11.805132 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:11.805206 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:11.805586 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.806354 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.807849 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.808275 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.808371 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.808421 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:11.808610 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 18:45:11.808668 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":56,"threads_started":1}
I20260806 18:45:11.808792 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.804459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35987 (local address 127.0.0.1:58982)
0806 18:45:11.804796 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.804812 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:11.804924 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:11.805330 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:11.805334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:11.805353 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:11.805455 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.805462 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.806461 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.806464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.807694 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.807705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.808387 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.808391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.808489 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.808515 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:45:11.808593 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 18:45:11.808663 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":54,"threads_started":1}
I20260806 18:45:11.809211 11560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9304d77916a4a1bacb81ddad68f9845" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c77cb7e640c048a0bc093a588c5ed78d" is_pre_election: true
I20260806 18:45:11.809315 12557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.805067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:45209 (local address 127.0.0.1:50004)
0806 18:45:11.805524 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.805537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:11.805610 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:11.805992 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:11.805995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:11.806012 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:11.806093 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.806098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.807170 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.807174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.808174 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.808185 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.808993 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.808997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.809081 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.809107 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:45:11.809157 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 18:45:11.809225 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":73,"threads_started":1}
I20260806 18:45:11.809420 11560 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9304d77916a4a1bacb81ddad68f9845 in term 1.
I20260806 18:45:11.809747 11795 leader_election.cc:302] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: b9304d77916a4a1bacb81ddad68f9845, c77cb7e640c048a0bc093a588c5ed78d; no voters: 246273a6f7bc49578d7729671bee2816
I20260806 18:45:11.809842 12558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.805255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50004 (local address 127.1.245.124:45209)
0806 18:45:11.805783 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:11.805787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:11.805799 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:45:11.805838 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:11.805841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:11.805879 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:11.806184 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:11.806263 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.807041 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.808291 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.808844 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.809554 (+ 710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.809608 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:11.809644 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 18:45:11.809711 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":91,"threads_started":1}
I20260806 18:45:11.809980 12549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 18:45:11.810039 12549 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 246273a6f7bc49578d7729671bee2816)
I20260806 18:45:11.810089 12549 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:45:11.810258 11820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c77cb7e640c048a0bc093a588c5ed78d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b9304d77916a4a1bacb81ddad68f9845" is_pre_election: true
I20260806 18:45:11.811328 12549 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } }
I20260806 18:45:11.811571 11820 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c77cb7e640c048a0bc093a588c5ed78d in current term 2: Already voted for candidate b9304d77916a4a1bacb81ddad68f9845 in this term.
I20260806 18:45:11.811712 12549 leader_election.cc:288] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [CANDIDATE]: Term 2 election: Requested vote from peers 246273a6f7bc49578d7729671bee2816 (127.1.245.126:37393), c77cb7e640c048a0bc093a588c5ed78d (127.1.245.125:35987)
I20260806 18:45:11.812040 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.811844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:35092)
0806 18:45:11.811929 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.811981 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 18:45:11.812120 11560 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b9304d77916a4a1bacb81ddad68f9845" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c77cb7e640c048a0bc093a588c5ed78d"
I20260806 18:45:11.812240 11560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:45:11.812403 11523 leader_election.cc:302] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c77cb7e640c048a0bc093a588c5ed78d; no voters: 246273a6f7bc49578d7729671bee2816, b9304d77916a4a1bacb81ddad68f9845
I20260806 18:45:11.813309 11560 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9304d77916a4a1bacb81ddad68f9845 in term 2.
I20260806 18:45:11.813427 12550 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:45:11.813562 11795 leader_election.cc:302] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9304d77916a4a1bacb81ddad68f9845, c77cb7e640c048a0bc093a588c5ed78d; no voters:
I20260806 18:45:11.813719 12549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:45:11.813915 12549 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [term 2 LEADER]: Becoming Leader. State: Replica: b9304d77916a4a1bacb81ddad68f9845, State: Running, Role: LEADER
I20260806 18:45:11.814041 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.813846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:35094)
0806 18:45:11.813932 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.813981 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 18:45:11.814165 11796 leader_election.cc:334] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 246273a6f7bc49578d7729671bee2816 (127.1.245.126:37393): Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
I20260806 18:45:11.814188 12549 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } }
I20260806 18:45:11.815393 12561 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9304d77916a4a1bacb81ddad68f9845. Latest consensus state: current_term: 2 leader_uuid: "b9304d77916a4a1bacb81ddad68f9845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:11.815502 12561 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:11.815775 12564 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:11.818100 12564 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:11.818547 12564 catalog_manager.cc:1259] Loaded cluster ID: 04423f5d736947abb745a66cce39e2f5
I20260806 18:45:11.818591 12564 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:11.819125 12564 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:11.819497 12564 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Loaded TSK: 0
I20260806 18:45:11.820173 12564 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:45:11.862914 11804 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:45:11.866128 11804 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:11.869144 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.868908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:35116)
0806 18:45:11.869018 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.869073 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 18:45:11.869283 11560 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [term 2 FOLLOWER]: Refusing update from remote peer b9304d77916a4a1bacb81ddad68f9845: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260806 18:45:11.869876 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.869626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.0.0.1:35132)
0806 18:45:11.869747 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.869803 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 18:45:11.869841 12561 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260806 18:45:11.870007 11796 consensus_peers.cc:487] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 -> Peer 246273a6f7bc49578d7729671bee2816 (127.1.245.126:37393): Couldn't send request to peer 246273a6f7bc49578d7729671bee2816. Status: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 18:45:11.871313 12550 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9304d77916a4a1bacb81ddad68f9845. Latest consensus state: current_term: 2 leader_uuid: "b9304d77916a4a1bacb81ddad68f9845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:11.871439 12550 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:45:11.872718 12561 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "b9304d77916a4a1bacb81ddad68f9845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:11.872838 12561 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:11.873154 12566 mvcc.cc:205] Tried to move back new op lower bound from 7315627671011741696 to 7315627670793265152. Current Snapshot: MvccSnapshot[applied={T|T < 7315627671011741696}]
I20260806 18:45:11.873287 12550 sys_catalog.cc:455] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "b9304d77916a4a1bacb81ddad68f9845" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "246273a6f7bc49578d7729671bee2816" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37393 } } peers { permanent_uuid: "c77cb7e640c048a0bc093a588c5ed78d" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35987 } attrs { promote: false } } peers { permanent_uuid: "b9304d77916a4a1bacb81ddad68f9845" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 45209 } attrs { promote: false } } }
I20260806 18:45:11.873389 12550 sys_catalog.cc:458] T 00000000000000000000000000000000 P c77cb7e640c048a0bc093a588c5ed78d [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:45:11.883213 12552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.878889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:37589 (local address 127.0.0.1:35554)
0806 18:45:11.879753 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.879770 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:11.879842 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:11.880127 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:11.880129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:11.880144 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:11.880220 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.880225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.881070 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.881074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.881795 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.881803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.882785 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.882789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.882883 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.882908 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:45:11.882973 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:45:11.883071 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":2}
I20260806 18:45:11.883445 12553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.878689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:41519 (local address 127.0.0.1:45602)
0806 18:45:11.878987 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.879000 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:11.879080 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:11.879526 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:11.879530 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:11.879551 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:11.879656 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.879663 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.880761 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.880764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.881909 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.881919 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.882998 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.883001 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.883137 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.883164 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:45:11.883240 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:45:11.883309 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":25}
I20260806 18:45:11.883620 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.879015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35554 (local address 127.1.245.66:37589)
0806 18:45:11.879314 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:11.879319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:11.879917 (+ 598us) server_negotiation.cc:407] Connection header received
0806 18:45:11.879978 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:11.879981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:11.880030 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:11.880110 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:11.880344 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.880975 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.881912 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.882558 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.882660 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.882727 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:11.883407 (+ 680us) server_negotiation.cc:299] Negotiation successful
0806 18:45:11.883486 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":69,"threads_started":1}
I20260806 18:45:11.883873 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.879388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:34405 (local address 127.0.0.1:34764)
0806 18:45:11.879831 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.879855 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:11.879930 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:11.880385 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:11.880388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:11.880405 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:11.880485 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.880491 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.881542 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.881545 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.882619 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:11.882628 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.883544 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.883548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:11.883632 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.883655 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:45:11.883715 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 18:45:11.883776 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":107,"threads_started":1}
I20260806 18:45:11.883935 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.878810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45602 (local address 127.1.245.65:41519)
0806 18:45:11.879275 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:11.879280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:11.879294 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:11.879343 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:11.879346 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:11.879396 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:11.879482 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:11.879792 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.880630 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.882037 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.882872 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.883078 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.883138 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:11.883262 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 18:45:11.883808 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":72,"threads_started":1}
I20260806 18:45:11.884145 12154 tablet_service.cc:1467] Processing CreateTablet for tablet 3b4c8ee71edf428b813fe8f02916b4f0 (DEFAULT_TABLE table=test-table [id=9f2fb87961dd445b809f5fdd0eef7f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:11.884298 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.879855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.1.245.67:34405)
0806 18:45:11.880169 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:11.880173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:11.880186 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:11.880234 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:11.880237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:11.880279 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:11.880355 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:11.880583 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.881417 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.882749 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:11.883339 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:11.883426 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:11.883488 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:11.884079 (+ 591us) server_negotiation.cc:299] Negotiation successful
0806 18:45:11.884159 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":77,"threads_started":1}
I20260806 18:45:11.884384 11969 tablet_service.cc:1467] Processing CreateTablet for tablet 3b4c8ee71edf428b813fe8f02916b4f0 (DEFAULT_TABLE table=test-table [id=9f2fb87961dd445b809f5fdd0eef7f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:11.884465 12154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b4c8ee71edf428b813fe8f02916b4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:11.884635 11969 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b4c8ee71edf428b813fe8f02916b4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:11.884755 12363 tablet_service.cc:1467] Processing CreateTablet for tablet 3b4c8ee71edf428b813fe8f02916b4f0 (DEFAULT_TABLE table=test-table [id=9f2fb87961dd445b809f5fdd0eef7f41]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:11.885013 12363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b4c8ee71edf428b813fe8f02916b4f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:11.887151 12582 tablet_bootstrap.cc:492] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32: Bootstrap starting.
I20260806 18:45:11.887686 12583 tablet_bootstrap.cc:492] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: Bootstrap starting.
I20260806 18:45:11.888119 12582 tablet_bootstrap.cc:654] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:11.888554 12583 tablet_bootstrap.cc:654] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:11.888836 12580 tablet_bootstrap.cc:492] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: Bootstrap starting.
I20260806 18:45:11.889473 12582 tablet_bootstrap.cc:492] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32: No bootstrap required, opened a new log
I20260806 18:45:11.889559 12582 ts_tablet_manager.cc:1397] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:11.889781 12583 tablet_bootstrap.cc:492] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: No bootstrap required, opened a new log
I20260806 18:45:11.889820 12580 tablet_bootstrap.cc:654] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:11.889858 12583 ts_tablet_manager.cc:1397] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:11.890053 12582 raft_consensus.cc:348] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:11.890163 12582 raft_consensus.cc:374] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:11.890197 12582 raft_consensus.cc:729] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea17dfec40434cd88bc6e868eb83fa32, State: Initialized, Role: FOLLOWER
I20260806 18:45:11.890295 12583 raft_consensus.cc:348] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:11.890462 12583 raft_consensus.cc:374] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:11.890496 12583 raft_consensus.cc:729] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fdb241e0e474cb3b1534857c34eaec7, State: Initialized, Role: FOLLOWER
I20260806 18:45:11.890445 12582 consensus_queue.cc:260] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:11.890647 12583 consensus_queue.cc:260] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:11.890784 12582 ts_tablet_manager.cc:1428] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:45:11.890969 12583 ts_tablet_manager.cc:1428] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:45:11.891232 12580 tablet_bootstrap.cc:492] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: No bootstrap required, opened a new log
I20260806 18:45:11.891319 12580 ts_tablet_manager.cc:1397] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:45:11.891767 12580 raft_consensus.cc:348] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:11.891892 12580 raft_consensus.cc:374] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:11.891927 12580 raft_consensus.cc:729] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76a9cf82ac3e485eb75e421bd6565b59, State: Initialized, Role: FOLLOWER
I20260806 18:45:11.892045 12580 consensus_queue.cc:260] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:11.892376 12580 ts_tablet_manager.cc:1428] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:11.892957 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.891847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.67:57821)
0806 18:45:11.892783 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.892846 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":871,"thread_start_us":95,"threads_started":1}
I20260806 18:45:11.893774 12454 heartbeater.cc:503] Master 127.1.245.124:45209 was elected leader, sending a full tablet report...
I20260806 18:45:11.894201 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.893352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.67:38759)
0806 18:45:11.894068 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.894121 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":89,"threads_started":1}
I20260806 18:45:11.895207 12256 heartbeater.cc:503] Master 127.1.245.124:45209 was elected leader, sending a full tablet report...
I20260806 18:45:11.896250 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.894057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.65:48757)
0806 18:45:11.896079 (+ 2022us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.896139 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1945,"thread_start_us":98,"threads_started":1}
I20260806 18:45:11.896679 12594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.894451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.66:52499)
0806 18:45:11.896558 (+ 2107us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.896604 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1964,"thread_start_us":89,"threads_started":1}
I20260806 18:45:11.897262 12594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.897080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.66:41723)
0806 18:45:11.897169 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:11.897212 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 18:45:11.998054 12587 raft_consensus.cc:482] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:11.998239 12587 raft_consensus.cc:504] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:11.999637 12587 leader_election.cc:288] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76a9cf82ac3e485eb75e421bd6565b59 (127.1.245.66:37589), 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405)
I20260806 18:45:12.004717 12575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.999970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:57821 (local address 127.1.245.66:37589)
0806 18:45:12.000085 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:12.000090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:12.000172 (+ 82us) server_negotiation.cc:407] Connection header received
0806 18:45:12.000361 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:12.000364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:12.000417 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:12.000497 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:12.000777 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.001565 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.003418 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.004046 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.004177 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:12.004233 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:12.004499 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:45:12.004577 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:45:12.004717 12578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.000027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:35077 (local address 127.1.245.67:34405)
0806 18:45:12.000162 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:12.000167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:12.000390 (+ 223us) server_negotiation.cc:407] Connection header received
0806 18:45:12.000468 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:12.000471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:12.000517 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:12.000585 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:12.000776 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.001567 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.003418 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.004046 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.004177 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:12.004233 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:12.004499 (+ 266us) server_negotiation.cc:299] Negotiation successful
0806 18:45:12.004577 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:45:12.004765 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.999838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:37589 (local address 127.1.245.65:57821)
0806 18:45:11.999985 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.000112 (+ 127us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:12.000250 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:12.000515 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:12.000517 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:12.000532 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:12.000667 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:12.000675 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.001686 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.001690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:12.003223 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:12.003237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.004205 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.004209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:12.004349 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:12.004394 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 18:45:12.004476 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 18:45:12.004588 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":207}
I20260806 18:45:12.004772 12596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:11.999860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:34405 (local address 127.1.245.65:35077)
0806 18:45:12.000308 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.000320 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:12.000399 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:12.000596 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:12.000599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:12.000611 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:12.000690 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:12.000695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.001686 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.001690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:12.003223 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:12.003237 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.004205 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.004209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:12.004395 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:12.004411 (+ 16us) client_negotiation.cc:769] Sending connection context
0806 18:45:12.004475 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:45:12.004588 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":214,"thread_start_us":152,"threads_started":1}
I20260806 18:45:12.005360 12393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b4c8ee71edf428b813fe8f02916b4f0" candidate_uuid: "ea17dfec40434cd88bc6e868eb83fa32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fdb241e0e474cb3b1534857c34eaec7" is_pre_election: true
I20260806 18:45:12.005539 12393 raft_consensus.cc:2454] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea17dfec40434cd88bc6e868eb83fa32 in term 0.
I20260806 18:45:12.005524 12179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b4c8ee71edf428b813fe8f02916b4f0" candidate_uuid: "ea17dfec40434cd88bc6e868eb83fa32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76a9cf82ac3e485eb75e421bd6565b59" is_pre_election: true
I20260806 18:45:12.005769 12179 raft_consensus.cc:2454] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea17dfec40434cd88bc6e868eb83fa32 in term 0.
I20260806 18:45:12.005977 11905 leader_election.cc:302] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3fdb241e0e474cb3b1534857c34eaec7, ea17dfec40434cd88bc6e868eb83fa32; no voters:
I20260806 18:45:12.006285 12587 raft_consensus.cc:2781] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:12.006361 12587 raft_consensus.cc:482] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:12.006386 12587 raft_consensus.cc:3037] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:12.007411 12587 raft_consensus.cc:504] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:12.007826 12587 leader_election.cc:288] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [CANDIDATE]: Term 1 election: Requested vote from peers 76a9cf82ac3e485eb75e421bd6565b59 (127.1.245.66:37589), 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405)
I20260806 18:45:12.008229 12393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b4c8ee71edf428b813fe8f02916b4f0" candidate_uuid: "ea17dfec40434cd88bc6e868eb83fa32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3fdb241e0e474cb3b1534857c34eaec7"
I20260806 18:45:12.008327 12179 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b4c8ee71edf428b813fe8f02916b4f0" candidate_uuid: "ea17dfec40434cd88bc6e868eb83fa32" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76a9cf82ac3e485eb75e421bd6565b59"
I20260806 18:45:12.008381 12393 raft_consensus.cc:3037] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:12.008440 12179 raft_consensus.cc:3037] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:12.009369 12179 raft_consensus.cc:2454] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea17dfec40434cd88bc6e868eb83fa32 in term 1.
I20260806 18:45:12.009377 12393 raft_consensus.cc:2454] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea17dfec40434cd88bc6e868eb83fa32 in term 1.
I20260806 18:45:12.009670 11907 leader_election.cc:302] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3fdb241e0e474cb3b1534857c34eaec7, 76a9cf82ac3e485eb75e421bd6565b59, ea17dfec40434cd88bc6e868eb83fa32; no voters:
I20260806 18:45:12.009858 12587 raft_consensus.cc:2781] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:12.010083 12587 raft_consensus.cc:686] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 1 LEADER]: Becoming Leader. State: Replica: ea17dfec40434cd88bc6e868eb83fa32, State: Running, Role: LEADER
I20260806 18:45:12.010375 12587 consensus_queue.cc:237] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } }
I20260806 18:45:12.010432 12052 heartbeater.cc:503] Master 127.1.245.124:45209 was elected leader, sending a full tablet report...
I20260806 18:45:12.011117 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.010874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.65:38087)
0806 18:45:12.010994 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.011050 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:45:12.012607 11814 catalog_manager.cc:5495] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea17dfec40434cd88bc6e868eb83fa32 (127.1.245.65). New cstate: current_term: 1 leader_uuid: "ea17dfec40434cd88bc6e868eb83fa32" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ea17dfec40434cd88bc6e868eb83fa32" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41519 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 } health_report { overall_health: UNKNOWN } } }
I20260806 18:45:12.038760 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.034593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45606 (local address 127.1.245.65:41519)
0806 18:45:12.034748 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:12.034753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:12.034768 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:45:12.034921 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:12.034925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:12.034993 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:12.035095 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:12.035312 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.036133 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.036974 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:12.037325 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:12.037420 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:12.037697 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:12.038373 (+ 676us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:12.038477 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:12.038558 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 18:45:12.038623 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:45:12.041779 12393 raft_consensus.cc:1261] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 1 FOLLOWER]: Refusing update from remote peer ea17dfec40434cd88bc6e868eb83fa32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:12.042037 12179 raft_consensus.cc:1261] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 FOLLOWER]: Refusing update from remote peer ea17dfec40434cd88bc6e868eb83fa32: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:12.042393 12587 consensus_queue.cc:1035] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3fdb241e0e474cb3b1534857c34eaec7" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 34405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:12.042661 12597 consensus_queue.cc:1035] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76a9cf82ac3e485eb75e421bd6565b59" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 37589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:12.044426 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.044066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.67:57529)
0806 18:45:12.044232 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.044298 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260806 18:45:12.044648 12594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.044362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.66:38393)
0806 18:45:12.044520 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.044574 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260806 18:45:12.048823 12605 mvcc.cc:205] Tried to move back new op lower bound from 7315627671717941248 to 7315627671596986368. Current Snapshot: MvccSnapshot[applied={T|T < 7315627671717941248 or (T in {7315627671717941248})}]
I20260806 18:45:12.049702 12592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.049430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.67:39023)
0806 18:45:12.049578 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.049634 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260806 18:45:12.051041 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.050727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.65:37445)
0806 18:45:12.050909 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.050969 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260806 18:45:12.051746 12594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.051323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.66:60721)
0806 18:45:12.051621 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.051676 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229}
I20260806 18:45:12.052222 12593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.052021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:37393 (local address 127.1.245.65:48355)
0806 18:45:12.052110 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.052157 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:37393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 18:45:12.057718 11816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:12.057900 11816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:12.061797 11816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b9304d77916a4a1bacb81ddad68f9845: Sending DeleteTablet for 3 replicas of tablet 3b4c8ee71edf428b813fe8f02916b4f0
I20260806 18:45:12.062542 12154 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b4c8ee71edf428b813fe8f02916b4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:12 UTC) from {username='oryx', principal='oryx/127.1.245.124@KRBTEST.COM'} at 127.0.0.1:35554
I20260806 18:45:12.062644 12363 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b4c8ee71edf428b813fe8f02916b4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:12 UTC) from {username='oryx', principal='oryx/127.1.245.124@KRBTEST.COM'} at 127.0.0.1:34764
I20260806 18:45:12.062737 11969 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b4c8ee71edf428b813fe8f02916b4f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:12 UTC) from {username='oryx', principal='oryx/127.1.245.124@KRBTEST.COM'} at 127.0.0.1:45602
I20260806 18:45:12.063215 12618 tablet_replica.cc:330] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32: stopping tablet replica
I20260806 18:45:12.063361 12618 raft_consensus.cc:2229] T 3b4c8ee71edf428b813fe8f02916b4f0 P ea17dfec40434cd88bc6e868eb83fa32 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:12.063513 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 11763
I20260806 18:45:12.068806 12616 tablet_replica.cc:330] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: stopping tablet replica
I20260806 18:45:12.068935 12615 tablet_replica.cc:330] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: stopping tablet replica
I20260806 18:45:12.068962 12616 raft_consensus.cc:2229] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:12.069058 12615 raft_consensus.cc:2229] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:12.069104 12616 raft_consensus.cc:2258] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:12.069188 12615 raft_consensus.cc:2258] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:12.070063 12616 ts_tablet_manager.cc:1905] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:12.070461 12615 ts_tablet_manager.cc:1905] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:12.072512 12615 ts_tablet_manager.cc:1918] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:12.072585 12615 log.cc:1196] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/3b4c8ee71edf428b813fe8f02916b4f0
I20260806 18:45:12.072816 12616 ts_tablet_manager.cc:1918] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:12.072841 12615 ts_tablet_manager.cc:1939] T 3b4c8ee71edf428b813fe8f02916b4f0 P 76a9cf82ac3e485eb75e421bd6565b59: Deleting consensus metadata
I20260806 18:45:12.072898 12616 log.cc:1196] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/3b4c8ee71edf428b813fe8f02916b4f0
I20260806 18:45:12.073143 12616 ts_tablet_manager.cc:1939] T 3b4c8ee71edf428b813fe8f02916b4f0 P 3fdb241e0e474cb3b1534857c34eaec7: Deleting consensus metadata
I20260806 18:45:12.073571 11798 catalog_manager.cc:4841] TS 76a9cf82ac3e485eb75e421bd6565b59 (127.1.245.66:37589): tablet 3b4c8ee71edf428b813fe8f02916b4f0 (table test-table [id=9f2fb87961dd445b809f5fdd0eef7f41]) successfully deleted
I20260806 18:45:12.073843 11796 catalog_manager.cc:4841] TS 3fdb241e0e474cb3b1534857c34eaec7 (127.1.245.67:34405): tablet 3b4c8ee71edf428b813fe8f02916b4f0 (table test-table [id=9f2fb87961dd445b809f5fdd0eef7f41]) successfully deleted
I20260806 18:45:12.076781 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 12064
I20260806 18:45:12.076828 12576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:12.076584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:41519 (local address 127.0.0.1:45608)
0806 18:45:12.076703 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:12.076753 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:41519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:45:12.078882 11798 catalog_manager.cc:4584] TS ea17dfec40434cd88bc6e868eb83fa32 (127.1.245.65:41519): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3b4c8ee71edf428b813fe8f02916b4f0: Network error: Client connection negotiation failed: client connection to 127.1.245.65:41519: connect: Connection refused (error 111)
I20260806 18:45:12.092581 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 12269
I20260806 18:45:12.104530 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 11348
I20260806 18:45:12.114979 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 11499
2026-08-06T18:45:12Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5738 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:45:12 dist-test-slave-fgb3 krb5kdc[12621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:45:12 dist-test-slave-fgb3 krb5kdc[12621](info): set up 2 sockets
Aug 06 18:45:12 dist-test-slave-fgb3 krb5kdc[12621](info): commencing operation
krb5kdc: starting...
W20260806 18:45:13.236168 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:45:13 dist-test-slave-fgb3 krb5kdc[12621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:45:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.126.keytab.
Entry for principal oryx/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.126.keytab.
I20260806 18:45:13.361857 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:41947
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.126.keytab
--principal=oryx/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:41947
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:13.476013 12636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:13.476262 12636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:13.476300 12636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:13.480324 12636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:13.480376 12636 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 18:45:13.480392 12636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:13.480409 12636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:13.480423 12636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:13.480439 12636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:13.485540 12636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:41947
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.126.keytab
--principal=oryx/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:41947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:13.486608 12636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:13.487633 12636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:13.495942 12644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:13.496034 12636 server_base.cc:1034] running on GCE node
W20260806 18:45:13.495985 12649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:13.495977 12642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:13.496660 12636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:13.497716 12636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:13.498872 12636 hybrid_clock.cc:648] HybridClock initialized: now 1786041913498851 us; error 35 us; skew 500 ppm
Aug 06 18:45:13 dist-test-slave-fgb3 krb5kdc[12621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041913, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:13.504941 12636 init.cc:377] Logged in from keytab as oryx/127.1.245.126@KRBTEST.COM (short username oryx)
I20260806 18:45:13.505818 12636 webserver.cc:460] Webserver started at http://127.1.245.126:37925/ using document root <none> and password file <none>
I20260806 18:45:13.506386 12636 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:13.506459 12636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:13.506659 12636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:13.508663 12636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553"
format_stamp: "Formatted at 2026-08-06 18:45:13 on dist-test-slave-fgb3"
server_key: "b8d6deedfd63031f36ed9f26ecfe9f04"
server_key_iv: "6678208358a4957682fd3531210d0545"
server_key_version: "encryptionkey@0"
I20260806 18:45:13.509186 12636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553"
format_stamp: "Formatted at 2026-08-06 18:45:13 on dist-test-slave-fgb3"
server_key: "b8d6deedfd63031f36ed9f26ecfe9f04"
server_key_iv: "6678208358a4957682fd3531210d0545"
server_key_version: "encryptionkey@0"
I20260806 18:45:13.512693 12636 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:45:13.514947 12657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:13.516079 12636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:13.516189 12636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553"
format_stamp: "Formatted at 2026-08-06 18:45:13 on dist-test-slave-fgb3"
server_key: "b8d6deedfd63031f36ed9f26ecfe9f04"
server_key_iv: "6678208358a4957682fd3531210d0545"
server_key_version: "encryptionkey@0"
I20260806 18:45:13.516275 12636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:13.534013 12636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:13.536795 12636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:13.536975 12636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:13.544754 12636 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:41947
I20260806 18:45:13.544766 12752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:41947 every 8 connection(s)
I20260806 18:45:13.545698 12636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:13.548297 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 12636
I20260806 18:45:13.548434 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:13.548727 12753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:13.548750 2005 external_mini_cluster.cc:1428] Setting key 92fcf4c7d74929351cc7b50cc6d4b52e
I20260806 18:45:13.555097 12753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: Bootstrap starting.
I20260806 18:45:13.557313 12753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: Neither blocks nor log segments found. Creating new log.
Aug 06 18:45:13 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:13.558184 12753 log.cc:824] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:13.560581 12753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: No bootstrap required, opened a new log
I20260806 18:45:13.564152 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:13.550254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42128 (local address 127.1.245.126:41947)
0806 18:45:13.550745 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:13.550762 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:13.550801 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:45:13.551524 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:13.551543 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:13.551919 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:13.552242 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:13.553121 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.554154 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.555051 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.555568 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.555744 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:13.558424 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:13.558431 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:13.558447 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:13.558479 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:13.560976 (+ 2497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:13.561453 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:13.561459 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:13.561465 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:13.561602 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:13.561850 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:13.561852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:13.561854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:13.562394 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:13.562598 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:13.562853 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 18:45:13.563164 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":141,"threads_started":1}
I20260806 18:45:13.564400 12753 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41947 } }
I20260806 18:45:13.564663 12753 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:13.564723 12753 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2eccca0d0d64b40aa4bf7c4e3cdf553, State: Initialized, Role: FOLLOWER
I20260806 18:45:13.565181 12753 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41947 } }
I20260806 18:45:13.565335 12753 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:13.565412 12753 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:13.565506 12753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:13.566457 12753 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41947 } }
I20260806 18:45:13.566777 12753 leader_election.cc:302] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2eccca0d0d64b40aa4bf7c4e3cdf553; no voters:
I20260806 18:45:13.567070 12753 leader_election.cc:288] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:13.567157 12761 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:13.567391 12761 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [term 1 LEADER]: Becoming Leader. State: Replica: d2eccca0d0d64b40aa4bf7c4e3cdf553, State: Running, Role: LEADER
I20260806 18:45:13.567684 12761 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41947 } }
I20260806 18:45:13.568221 12753 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:13.569289 12764 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2eccca0d0d64b40aa4bf7c4e3cdf553. Latest consensus state: current_term: 1 leader_uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41947 } } }
I20260806 18:45:13.569416 12764 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:13.569666 12762 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2eccca0d0d64b40aa4bf7c4e3cdf553" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 41947 } } }
I20260806 18:45:13.569761 12762 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:13.569890 12772 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:13.573082 12772 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:13.578940 12772 catalog_manager.cc:1347] Generated new cluster ID: 6150241048834be8bcc5a0ba3f9e8cb7
I20260806 18:45:13.579008 12772 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:13.599185 12772 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:13.600142 12772 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:13.608287 12772 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: Generated new TSK 0
I20260806 18:45:13.609026 12772 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab.
Entry for principal oryx/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab.
I20260806 18:45:13.643637 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41947
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab
--principal=oryx/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:13.777952 12790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:13.778276 12790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:13.778365 12790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:13.782447 12790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:13.782524 12790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:13.782569 12790 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:45:13.782630 12790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:13.788062 12790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab
--principal=oryx/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:41947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:13.789228 12790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:13.790405 12790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:13.800267 12797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:13.800271 12801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:13.800534 12790 server_base.cc:1034] running on GCE node
W20260806 18:45:13.800267 12798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:13.801098 12790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:13.801790 12790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:13.802955 12790 hybrid_clock.cc:648] HybridClock initialized: now 1786041913802931 us; error 46 us; skew 500 ppm
Aug 06 18:45:13 dist-test-slave-fgb3 krb5kdc[12621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041913, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:13.805768 12790 init.cc:377] Logged in from keytab as oryx/127.1.245.65@KRBTEST.COM (short username oryx)
I20260806 18:45:13.806712 12790 webserver.cc:460] Webserver started at http://127.1.245.65:33239/ using document root <none> and password file <none>
I20260806 18:45:13.807263 12790 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:13.807331 12790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:13.807559 12790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:13.809612 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "9a0dfc72c6fe460eb36872748f728337"
format_stamp: "Formatted at 2026-08-06 18:45:13 on dist-test-slave-fgb3"
server_key: "6043e3af13f85189402cc4f965b95ce5"
server_key_iv: "2244a6882cbff1fe50077e6cef79d4f1"
server_key_version: "encryptionkey@0"
I20260806 18:45:13.810140 12790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "9a0dfc72c6fe460eb36872748f728337"
format_stamp: "Formatted at 2026-08-06 18:45:13 on dist-test-slave-fgb3"
server_key: "6043e3af13f85189402cc4f965b95ce5"
server_key_iv: "2244a6882cbff1fe50077e6cef79d4f1"
server_key_version: "encryptionkey@0"
I20260806 18:45:13.813925 12790 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:45:13.816478 12815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:13.817649 12790 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:13.817760 12790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "9a0dfc72c6fe460eb36872748f728337"
format_stamp: "Formatted at 2026-08-06 18:45:13 on dist-test-slave-fgb3"
server_key: "6043e3af13f85189402cc4f965b95ce5"
server_key_iv: "2244a6882cbff1fe50077e6cef79d4f1"
server_key_version: "encryptionkey@0"
I20260806 18:45:13.817857 12790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:13.847427 12790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:13.850863 12790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:13.851071 12790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:13.852626 12790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:13.852679 12790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:13.852723 12790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:13.852756 12790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:45:13 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041913, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:13.863204 12790 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:38933
I20260806 18:45:13.863231 12977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:38933 every 8 connection(s)
I20260806 18:45:13.864151 12790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:13.867369 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:13.853567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:49685 (local address 127.1.245.126:41947)
0806 18:45:13.853687 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:13.853691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:13.854841 (+ 1150us) server_negotiation.cc:407] Connection header received
0806 18:45:13.856062 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:13.856066 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:13.856128 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:13.856217 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:13.857861 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.858639 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.860336 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.860569 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.860656 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:13.863612 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:13.863616 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:13.863619 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:13.863648 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:13.865285 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:13.865856 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:13.865859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:13.865861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:13.865950 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:13.866286 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:13.866288 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:13.866289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:13.866523 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:13.866603 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:13.867094 (+ 491us) server_negotiation.cc:299] Negotiation successful
0806 18:45:13.867208 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:45:13.868122 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:13.854123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41947 (local address 127.1.245.65:49685)
0806 18:45:13.854666 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:13.854711 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:13.855827 (+ 1116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:13.856458 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:13.856468 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:13.856927 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:13.857671 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:13.857694 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.858778 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.858782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:13.860217 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:13.860231 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.860734 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.860739 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:13.860939 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:13.861621 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:13.861644 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:13.863446 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:13.865426 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:13.865431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:13.865442 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:13.865714 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:13.866046 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:13.866048 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:13.866050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:13.866184 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:13.866625 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:13.866631 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:13.866877 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 18:45:13.867045 (+ 168us) client_negotiation.cc:240] Negotiation successful
0806 18:45:13.867293 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":185,"threads_started":1}
I20260806 18:45:13.869470 12978 heartbeater.cc:347] Connected to a master server at 127.1.245.126:41947
I20260806 18:45:13.869612 12978 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:13.870155 12978 heartbeater.cc:511] Master 127.1.245.126:41947 requested a full tablet report, sending...
I20260806 18:45:13.871692 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 12790
I20260806 18:45:13.871819 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:45:13.871845 12687 ts_manager.cc:194] Registered new tserver with Master: 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933)
I20260806 18:45:13.872131 2005 external_mini_cluster.cc:1428] Setting key 4a69c98539d27ba36a06eed34f9376cf
I20260806 18:45:13.873538 12687 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:49685
I20260806 18:45:13.878588 12985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:13.871567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41947 (local address 127.1.245.65:37557)
0806 18:45:13.871843 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:13.871858 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:13.871952 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:13.872166 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:13.872169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:13.872359 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:13.872460 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:13.872467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.873496 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.873499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:13.874156 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:13.874168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.874702 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.874706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:13.874814 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:13.875336 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:13.875350 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:13.875777 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:13.877186 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:13.877189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:13.877192 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:13.877382 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:13.877639 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:13.877642 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:13.877644 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:13.877724 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:13.878075 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:13.878078 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:13.878168 (+ 90us) client_negotiation.cc:769] Sending connection context
0806 18:45:13.878277 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:45:13.878433 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":98,"threads_started":1}
I20260806 18:45:13.878677 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:13.871627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:37557 (local address 127.1.245.126:41947)
0806 18:45:13.871843 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:13.871847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:13.871920 (+ 73us) server_negotiation.cc:407] Connection header received
0806 18:45:13.872033 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:13.872036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:13.872077 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:13.872139 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:13.872592 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.873355 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.874289 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:13.874580 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:13.874684 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:13.875902 (+ 1218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:13.875906 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:13.875909 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:13.875934 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:13.877085 (+ 1151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:13.877472 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:13.877475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:13.877478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:13.877556 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:13.877804 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:13.877807 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:13.877809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:13.877985 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:13.878070 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:13.878399 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 18:45:13.878524 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":76,"server-negotiator.queue_time_us":147}
WARNING: no policy specified for oryx/127.1.245.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab.
Entry for principal oryx/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab.
I20260806 18:45:13.901027 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41947
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab
--principal=oryx/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:14.012409 12988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:14.012667 12988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:14.012735 12988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:14.017050 12988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:14.017117 12988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:14.017174 12988 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:45:14.017226 12988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:14.022799 12988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab
--principal=oryx/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:41947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:14.023913 12988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:14.024971 12988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:14.034631 12994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:14.034622 12997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:14.034622 12993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:14.034849 12988 server_base.cc:1034] running on GCE node
I20260806 18:45:14.035346 12988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:14.035979 12988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:14.037148 12988 hybrid_clock.cc:648] HybridClock initialized: now 1786041914037124 us; error 33 us; skew 500 ppm
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041914, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:14.040071 12988 init.cc:377] Logged in from keytab as oryx/127.1.245.66@KRBTEST.COM (short username oryx)
I20260806 18:45:14.041023 12988 webserver.cc:460] Webserver started at http://127.1.245.66:39903/ using document root <none> and password file <none>
I20260806 18:45:14.041590 12988 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:14.041658 12988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:14.041862 12988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:14.043977 12988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "edff284cbe514cdb99e162e92d303825"
format_stamp: "Formatted at 2026-08-06 18:45:14 on dist-test-slave-fgb3"
server_key: "47e11ab3a4ebceaddd1d9aaf5b5d1776"
server_key_iv: "1377af8e352c848424895d67d9f81051"
server_key_version: "encryptionkey@0"
I20260806 18:45:14.044541 12988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "edff284cbe514cdb99e162e92d303825"
format_stamp: "Formatted at 2026-08-06 18:45:14 on dist-test-slave-fgb3"
server_key: "47e11ab3a4ebceaddd1d9aaf5b5d1776"
server_key_iv: "1377af8e352c848424895d67d9f81051"
server_key_version: "encryptionkey@0"
I20260806 18:45:14.048319 12988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:45:14.050880 13008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:14.052102 12988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:14.052217 12988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "edff284cbe514cdb99e162e92d303825"
format_stamp: "Formatted at 2026-08-06 18:45:14 on dist-test-slave-fgb3"
server_key: "47e11ab3a4ebceaddd1d9aaf5b5d1776"
server_key_iv: "1377af8e352c848424895d67d9f81051"
server_key_version: "encryptionkey@0"
I20260806 18:45:14.052350 12988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:14.066346 12988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:14.069330 12988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:14.069525 12988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:14.071085 12988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:14.071138 12988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:14.071192 12988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:14.071220 12988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041914, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:14.081660 12988 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:42777
I20260806 18:45:14.081668 13159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:42777 every 8 connection(s)
I20260806 18:45:14.082796 12988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:14.084249 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.071746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:48557 (local address 127.1.245.126:41947)
0806 18:45:14.071897 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.071903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.072790 (+ 887us) server_negotiation.cc:407] Connection header received
0806 18:45:14.073950 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.073954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.074015 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.074108 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:14.075659 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.076226 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.077475 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.077754 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.077849 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.080569 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:14.080573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:14.080576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:14.080609 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:14.082190 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.082673 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.082677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.082679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.082791 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.083142 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.083145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.083147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.083387 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:14.083488 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.084018 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.084127 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:45:14.084959 13019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.072122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41947 (local address 127.1.245.66:48557)
0806 18:45:14.072612 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.072669 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.073707 (+ 1038us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.074340 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.074351 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.074839 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:14.075506 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.075526 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.076350 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.076354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.077359 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.077370 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.077914 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.077918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.078080 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.078613 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:14.078634 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:14.080389 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:14.082295 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.082299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.082331 (+ 32us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.082588 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.082878 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.082880 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.082883 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.083028 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.083543 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:14.083549 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:14.083782 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.083934 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.084184 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":107,"threads_started":1}
I20260806 18:45:14.086165 13160 heartbeater.cc:347] Connected to a master server at 127.1.245.126:41947
I20260806 18:45:14.086304 13160 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:14.086817 13160 heartbeater.cc:511] Master 127.1.245.126:41947 requested a full tablet report, sending...
I20260806 18:45:14.087435 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 12988
I20260806 18:45:14.087545 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:45:14.087810 2005 external_mini_cluster.cc:1428] Setting key 6dcb30998ec1e487f737b08571773d5c
I20260806 18:45:14.087929 12687 ts_manager.cc:194] Registered new tserver with Master: edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777)
I20260806 18:45:14.088687 12687 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:48557
I20260806 18:45:14.096117 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.088514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:45987 (local address 127.1.245.126:41947)
0806 18:45:14.088623 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.088627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.089017 (+ 390us) server_negotiation.cc:407] Connection header received
0806 18:45:14.089176 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.089179 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.089229 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.089311 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:14.089874 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.090639 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.091479 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.091801 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.091903 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.093109 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:14.093113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:14.093114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:14.093141 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:14.094572 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.094976 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.094980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.094982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.095057 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.095299 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.095302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.095304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.095515 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:14.095603 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.095850 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.095955 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:45:14.096120 13169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.088666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41947 (local address 127.1.245.66:45987)
0806 18:45:14.088939 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.088954 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.089079 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.089421 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.089424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.089627 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:14.089744 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.089751 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.090801 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.090805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.091366 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.091374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.091900 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.091902 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.091968 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.092467 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:14.092481 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:14.092812 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:14.094681 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.094684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.094686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.094887 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.095138 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.095141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.095143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.095221 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.095609 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:14.095612 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:14.095698 (+ 86us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.095782 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.095932 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":99,"threads_started":1}
WARNING: no policy specified for oryx/127.1.245.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab.
Entry for principal oryx/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab.
I20260806 18:45:14.115372 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:41947
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab
--principal=oryx/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:14.229704 13175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:14.229987 13175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:14.230049 13175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:14.234261 13175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:14.234350 13175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:14.234408 13175 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 18:45:14.234462 13175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:14.239776 13175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab
--principal=oryx/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.245.126:41947
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:14.240907 13175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:14.241978 13175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:14.251612 13182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:14.251618 13184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:14.251636 13186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:14.251762 13175 server_base.cc:1034] running on GCE node
I20260806 18:45:14.252254 13175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:14.252931 13175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:14.254109 13175 hybrid_clock.cc:648] HybridClock initialized: now 1786041914254080 us; error 42 us; skew 500 ppm
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041914, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:14.257025 13175 init.cc:377] Logged in from keytab as oryx/127.1.245.67@KRBTEST.COM (short username oryx)
I20260806 18:45:14.258021 13175 webserver.cc:460] Webserver started at http://127.1.245.67:37907/ using document root <none> and password file <none>
I20260806 18:45:14.258646 13175 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:14.258719 13175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:14.258945 13175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:14.261051 13175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "e583f83ff95447a0bd7a82de3b9a8e85"
format_stamp: "Formatted at 2026-08-06 18:45:14 on dist-test-slave-fgb3"
server_key: "33392accc60467a2b26147798bf8f2d6"
server_key_iv: "5feea1a2c82048383f8190e95479a502"
server_key_version: "encryptionkey@0"
I20260806 18:45:14.261619 13175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "e583f83ff95447a0bd7a82de3b9a8e85"
format_stamp: "Formatted at 2026-08-06 18:45:14 on dist-test-slave-fgb3"
server_key: "33392accc60467a2b26147798bf8f2d6"
server_key_iv: "5feea1a2c82048383f8190e95479a502"
server_key_version: "encryptionkey@0"
I20260806 18:45:14.265448 13175 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:45:14.268218 13195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:14.269654 13175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:45:14.269767 13175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "e583f83ff95447a0bd7a82de3b9a8e85"
format_stamp: "Formatted at 2026-08-06 18:45:14 on dist-test-slave-fgb3"
server_key: "33392accc60467a2b26147798bf8f2d6"
server_key_iv: "5feea1a2c82048383f8190e95479a502"
server_key_version: "encryptionkey@0"
I20260806 18:45:14.269877 13175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:14.295083 13175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:14.298417 13175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:14.298614 13175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:14.300140 13175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:14.300194 13175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:14.300248 13175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:14.300278 13175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:14.310496 13175 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:38729
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041914, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:14.310532 13336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:38729 every 8 connection(s)
I20260806 18:45:14.311570 13175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:14.312314 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 13175
I20260806 18:45:14.312461 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:14.312767 2005 external_mini_cluster.cc:1428] Setting key 191300e6ec2e4d88984b6d53a1d2d8fc
I20260806 18:45:14.315160 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.301082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:59513 (local address 127.1.245.126:41947)
0806 18:45:14.301203 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.301207 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.302390 (+ 1183us) server_negotiation.cc:407] Connection header received
0806 18:45:14.303462 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.303466 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.303528 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.303606 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:14.305139 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.305966 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.307716 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.308058 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.308184 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.311331 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:14.311335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:14.311338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:14.311370 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:14.312937 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.313498 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.313501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.313502 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.313602 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.313984 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.313987 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.313988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.314234 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:14.314352 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.314870 (+ 518us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.315001 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:45:14.315989 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.301617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41947 (local address 127.1.245.67:59513)
0806 18:45:14.302172 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.302217 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.303227 (+ 1010us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.303816 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.303825 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.304254 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:14.304946 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.304971 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.306109 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.306113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.307579 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.307590 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.308287 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.308292 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.308506 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.309217 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:14.309239 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:14.311071 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:14.313050 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.313056 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.313073 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.313387 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.313697 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.313700 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.313702 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.313865 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.314367 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:14.314374 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:14.314623 (+ 249us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.314787 (+ 164us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.315127 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":149,"threads_started":1}
I20260806 18:45:14.317168 13338 heartbeater.cc:347] Connected to a master server at 127.1.245.126:41947
I20260806 18:45:14.317314 13338 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:14.317909 13338 heartbeater.cc:511] Master 127.1.245.126:41947 requested a full tablet report, sending...
I20260806 18:45:14.319057 12687 ts_manager.cc:194] Registered new tserver with Master: e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729)
I20260806 18:45:14.319782 12687 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:59513
I20260806 18:45:14.326427 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.319511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41947 (local address 127.1.245.67:59141)
0806 18:45:14.319831 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.319847 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.319961 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.320214 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.320217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.320444 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:14.320563 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.320571 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.321450 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.321454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.322111 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.322120 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.322626 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.322628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.322697 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.323173 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:14.323187 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:14.323522 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:14.324931 (+ 1409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.324933 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.324935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.325137 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.325429 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.325432 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.325434 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.325528 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.325916 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:14.325919 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:14.326002 (+ 83us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.326088 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.326225 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":147,"threads_started":1}
I20260806 18:45:14.326445 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.319605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:59141 (local address 127.1.245.126:41947)
0806 18:45:14.319718 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.319723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.319921 (+ 198us) server_negotiation.cc:407] Connection header received
0806 18:45:14.320050 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.320053 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.320093 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.320155 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:14.320703 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.321324 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.322233 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.322522 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.322599 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.323638 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:14.323641 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:14.323644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:14.323667 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:14.324839 (+ 1172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.325220 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.325223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.325224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.325319 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.325603 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.325606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.325607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.325827 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:14.325918 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.326157 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.326272 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:45:14.326851 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:45:14.351176 12759 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:14.344985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49910 (local address 127.1.245.126:41947)
0806 18:45:14.345131 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.345134 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.345147 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:14.345188 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.345191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.345232 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.345295 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:14.346011 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.346595 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.347490 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.347699 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.347774 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.351035 (+ 3261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49910: BlockingRecv error: recv got EOF from 127.0.0.1:49910 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:14.360494 12759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.352606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49918 (local address 127.1.245.126:41947)
0806 18:45:14.352716 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.352720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.352839 (+ 119us) server_negotiation.cc:407] Connection header received
0806 18:45:14.352941 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.352943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.352983 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.353050 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:14.353707 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.354203 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.355029 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.355234 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.355340 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.357819 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:14.357822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:14.357825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:14.357850 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:14.359239 (+ 1389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.359600 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.359602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.359603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.359669 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.359857 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.359858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.359859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.360016 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:14.360110 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.360255 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.360343 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:45:14.363446 12687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:45:14.366540 12687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:14.381639 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.376112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57206 (local address 127.1.245.65:38933)
0806 18:45:14.376868 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.376874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.376898 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:45:14.376971 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.376976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.377160 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.377568 (+ 408us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:14.377746 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.378936 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.380136 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.380894 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.381002 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.381151 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.381386 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.381500 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":90,"threads_started":1}
I20260806 18:45:14.381755 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.375281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:38729 (local address 127.0.0.1:40474)
0806 18:45:14.375755 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.375809 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.375941 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.377291 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.377297 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.377333 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:14.377656 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.377664 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.379226 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.379230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.380402 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.380421 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.381395 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.381399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.381495 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.381518 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.381567 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.381631 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":202,"spinlock_wait_cycles":5760,"thread_start_us":93,"threads_started":1}
I20260806 18:45:14.381922 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.376013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:42777 (local address 127.0.0.1:48600)
0806 18:45:14.376853 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.376867 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.376963 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.377404 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.377407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.377424 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:14.377625 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.377637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.378799 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.378803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.380286 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.380298 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.381548 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.381551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.381631 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.381654 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.381715 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.381786 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":126,"spinlock_wait_cycles":1280,"thread_start_us":118,"threads_started":1}
I20260806 18:45:14.381598 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.375374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:38933 (local address 127.0.0.1:57206)
0806 18:45:14.376274 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.376286 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.376366 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.377291 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.377306 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.377334 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:14.377628 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.377637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.379073 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.379076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.380019 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.380029 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.381022 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.381025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.381142 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.381182 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.381296 (+ 114us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.381431 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":125,"spinlock_wait_cycles":6784,"thread_start_us":87,"threads_started":1}
I20260806 18:45:14.382426 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.376485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48600 (local address 127.1.245.66:42777)
0806 18:45:14.376796 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.376809 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.376944 (+ 135us) server_negotiation.cc:407] Connection header received
0806 18:45:14.377057 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.377061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.377209 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.377326 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:14.377768 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.378679 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.380430 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.381008 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.381104 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.382107 (+ 1003us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.382159 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.382249 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":102,"threads_started":1}
I20260806 18:45:14.382974 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.376463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.1.245.67:38729)
0806 18:45:14.376796 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.376809 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.376833 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:45:14.376907 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.376912 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.377124 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.377288 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:14.377803 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.379099 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.380523 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.381260 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.382393 (+ 1133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.382700 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.382751 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.382837 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":119,"threads_started":1}
I20260806 18:45:14.385142 13072 tablet_service.cc:1467] Processing CreateTablet for tablet 3f45c90500a2436ebd1f11ac8886a99d (DEFAULT_TABLE table=test-table [id=de8eb48f10834e75b71aaf0434332b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:14.385639 13257 tablet_service.cc:1467] Processing CreateTablet for tablet 3f45c90500a2436ebd1f11ac8886a99d (DEFAULT_TABLE table=test-table [id=de8eb48f10834e75b71aaf0434332b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:14.385640 12884 tablet_service.cc:1467] Processing CreateTablet for tablet 3f45c90500a2436ebd1f11ac8886a99d (DEFAULT_TABLE table=test-table [id=de8eb48f10834e75b71aaf0434332b2b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:14.386300 13072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f45c90500a2436ebd1f11ac8886a99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.386682 13257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f45c90500a2436ebd1f11ac8886a99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.386682 12884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f45c90500a2436ebd1f11ac8886a99d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.393189 13377 tablet_bootstrap.cc:492] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Bootstrap starting.
I20260806 18:45:14.394636 13378 tablet_bootstrap.cc:492] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Bootstrap starting.
I20260806 18:45:14.394636 13380 tablet_bootstrap.cc:492] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Bootstrap starting.
I20260806 18:45:14.395660 13377 tablet_bootstrap.cc:654] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.396389 13380 tablet_bootstrap.cc:654] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.396574 13378 tablet_bootstrap.cc:654] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.396569 13377 log.cc:824] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:14.397192 13380 log.cc:824] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:14.397365 13378 log.cc:824] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:14.398690 13377 tablet_bootstrap.cc:492] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: No bootstrap required, opened a new log
I20260806 18:45:14.398890 13377 ts_tablet_manager.cc:1397] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260806 18:45:14.399039 13378 tablet_bootstrap.cc:492] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: No bootstrap required, opened a new log
I20260806 18:45:14.399147 13380 tablet_bootstrap.cc:492] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: No bootstrap required, opened a new log
I20260806 18:45:14.399216 13378 ts_tablet_manager.cc:1397] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:45:14.399317 13380 ts_tablet_manager.cc:1397] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:45:14.402561 13378 raft_consensus.cc:348] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.402802 13378 raft_consensus.cc:374] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.402858 13378 raft_consensus.cc:729] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0dfc72c6fe460eb36872748f728337, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.402745 13377 raft_consensus.cc:348] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.402997 13377 raft_consensus.cc:374] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.403049 13377 raft_consensus.cc:729] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edff284cbe514cdb99e162e92d303825, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.402953 13380 raft_consensus.cc:348] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.403178 13380 raft_consensus.cc:374] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.403239 13380 raft_consensus.cc:729] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e583f83ff95447a0bd7a82de3b9a8e85, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.403352 13378 consensus_queue.cc:260] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.403502 13377 consensus_queue.cc:260] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.403610 13380 consensus_queue.cc:260] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.404381 13378 ts_tablet_manager.cc:1428] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:45:14.404430 13380 ts_tablet_manager.cc:1428] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:45:14.404439 13377 ts_tablet_manager.cc:1428] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260806 18:45:14.404749 13338 heartbeater.cc:503] Master 127.1.245.126:41947 was elected leader, sending a full tablet report...
I20260806 18:45:14.404749 13160 heartbeater.cc:503] Master 127.1.245.126:41947 was elected leader, sending a full tablet report...
I20260806 18:45:14.404750 12978 heartbeater.cc:503] Master 127.1.245.126:41947 was elected leader, sending a full tablet report...
W20260806 18:45:14.562870 13340 tablet.cc:2369] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:45:14.584075 13164 tablet.cc:2369] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:45:14.615680 12979 tablet.cc:2369] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:45:14.632406 13389 raft_consensus.cc:482] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:14.632611 13389 raft_consensus.cc:504] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.633734 13389 leader_election.cc:288] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933), edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777)
I20260806 18:45:14.637645 13207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.634063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:38933 (local address 127.1.245.67:57831)
0806 18:45:14.634205 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.634221 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.634358 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.634719 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.634723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.634758 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:14.634885 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.634892 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.635669 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.635672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.636666 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.636684 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.637323 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.637325 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.637404 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.637425 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.637481 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.637536 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:45:14.637714 13373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.634136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:57831 (local address 127.1.245.65:38933)
0806 18:45:14.634266 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.634272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.634289 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:45:14.634473 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.634477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.634521 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.634589 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:14.634985 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.635556 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.636802 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.637212 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.637330 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.637398 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.637506 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.637580 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:45:14.638350 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.634504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:50281 (local address 127.1.245.66:42777)
0806 18:45:14.634653 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.634657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.634828 (+ 171us) server_negotiation.cc:407] Connection header received
0806 18:45:14.634928 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.634930 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.634976 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.635052 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:14.635721 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.636196 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.637288 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.637807 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.637929 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.637989 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.638174 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.638232 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:45:14.638391 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.634413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:42777 (local address 127.1.245.67:50281)
0806 18:45:14.634729 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.634750 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.634850 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.635100 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.635103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.635121 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:14.635593 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.635601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.636307 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.636310 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.637166 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.637175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.637937 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.637940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.638057 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.638081 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.638154 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.638216 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260806 18:45:14.638425 12914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f45c90500a2436ebd1f11ac8886a99d" candidate_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0dfc72c6fe460eb36872748f728337" is_pre_election: true
I20260806 18:45:14.638811 12914 raft_consensus.cc:2454] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e583f83ff95447a0bd7a82de3b9a8e85 in term 0.
I20260806 18:45:14.639034 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f45c90500a2436ebd1f11ac8886a99d" candidate_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff284cbe514cdb99e162e92d303825" is_pre_election: true
I20260806 18:45:14.639303 13091 raft_consensus.cc:2454] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e583f83ff95447a0bd7a82de3b9a8e85 in term 0.
I20260806 18:45:14.639303 13196 leader_election.cc:302] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a0dfc72c6fe460eb36872748f728337, e583f83ff95447a0bd7a82de3b9a8e85; no voters:
I20260806 18:45:14.639592 13389 raft_consensus.cc:2781] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:14.639699 13389 raft_consensus.cc:482] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:14.639755 13389 raft_consensus.cc:3037] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:14.640796 13389 raft_consensus.cc:504] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.641111 13389 leader_election.cc:288] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [CANDIDATE]: Term 1 election: Requested vote from peers 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933), edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777)
I20260806 18:45:14.641463 12914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f45c90500a2436ebd1f11ac8886a99d" candidate_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0dfc72c6fe460eb36872748f728337"
I20260806 18:45:14.641481 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f45c90500a2436ebd1f11ac8886a99d" candidate_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff284cbe514cdb99e162e92d303825"
I20260806 18:45:14.641593 12914 raft_consensus.cc:3037] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:14.641602 13091 raft_consensus.cc:3037] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:14.642839 12914 raft_consensus.cc:2454] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e583f83ff95447a0bd7a82de3b9a8e85 in term 1.
I20260806 18:45:14.642839 13091 raft_consensus.cc:2454] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e583f83ff95447a0bd7a82de3b9a8e85 in term 1.
I20260806 18:45:14.643142 13196 leader_election.cc:302] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e583f83ff95447a0bd7a82de3b9a8e85, edff284cbe514cdb99e162e92d303825; no voters:
I20260806 18:45:14.643292 13389 raft_consensus.cc:2781] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:14.643527 13389 raft_consensus.cc:686] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 LEADER]: Becoming Leader. State: Replica: e583f83ff95447a0bd7a82de3b9a8e85, State: Running, Role: LEADER
I20260806 18:45:14.643826 13389 consensus_queue.cc:237] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } }
I20260806 18:45:14.646993 12683 catalog_manager.cc:5495] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67). New cstate: current_term: 1 leader_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } health_report { overall_health: UNKNOWN } } }
I20260806 18:45:14.674021 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.670425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40486 (local address 127.1.245.67:38729)
0806 18:45:14.670535 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.670539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.670552 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:14.670656 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.670658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.670714 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.670809 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:14.671071 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.671597 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.672484 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.672761 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.672850 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.673112 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:14.673641 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:14.673756 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.673847 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.673901 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:45:14.679958 13091 raft_consensus.cc:1261] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Refusing update from remote peer e583f83ff95447a0bd7a82de3b9a8e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:14.679958 12914 raft_consensus.cc:1261] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Refusing update from remote peer e583f83ff95447a0bd7a82de3b9a8e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:14.680672 13389 consensus_queue.cc:1035] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:14.680899 13391 consensus_queue.cc:1035] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:14.688556 13398 mvcc.cc:205] Tried to move back new op lower bound from 7315627682521550848 to 7315627682383450112. Current Snapshot: MvccSnapshot[applied={T|T < 7315627682521550848}]
I20260806 18:45:14.688654 13396 mvcc.cc:205] Tried to move back new op lower bound from 7315627682521550848 to 7315627682383450112. Current Snapshot: MvccSnapshot[applied={T|T < 7315627682521550848}]
I20260806 18:45:14.696564 12686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:14.696764 12686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:14.699445 12686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: Sending DeleteTablet for 3 replicas of tablet 3f45c90500a2436ebd1f11ac8886a99d
I20260806 18:45:14.700260 13257 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f45c90500a2436ebd1f11ac8886a99d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:14 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:40474
I20260806 18:45:14.700318 13072 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f45c90500a2436ebd1f11ac8886a99d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:14 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:48600
I20260806 18:45:14.700438 12884 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f45c90500a2436ebd1f11ac8886a99d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:14 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:57206
I20260806 18:45:14.700758 13407 tablet_replica.cc:330] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: stopping tablet replica
I20260806 18:45:14.701094 13408 tablet_replica.cc:330] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: stopping tablet replica
I20260806 18:45:14.701092 13407 raft_consensus.cc:2229] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:14.701465 13408 raft_consensus.cc:2229] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:14.701468 13407 raft_consensus.cc:2258] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:14.701545 12686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:45:14.701974 12686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:14.702171 13408 raft_consensus.cc:2258] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:14.702639 13410 tablet_replica.cc:330] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: stopping tablet replica
I20260806 18:45:14.703052 13410 raft_consensus.cc:2229] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:14.703117 13407 ts_tablet_manager.cc:1905] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:14.703461 13410 raft_consensus.cc:2258] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:14.704885 13410 ts_tablet_manager.cc:1905] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:14.705271 13408 ts_tablet_manager.cc:1905] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:14.706003 13407 ts_tablet_manager.cc:1918] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:14.706104 13407 log.cc:1196] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/3f45c90500a2436ebd1f11ac8886a99d
I20260806 18:45:14.706456 13407 ts_tablet_manager.cc:1939] T 3f45c90500a2436ebd1f11ac8886a99d P edff284cbe514cdb99e162e92d303825: Deleting consensus metadata
I20260806 18:45:14.707784 12658 catalog_manager.cc:4841] TS edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777): tablet 3f45c90500a2436ebd1f11ac8886a99d (table test-table [id=de8eb48f10834e75b71aaf0434332b2b]) successfully deleted
I20260806 18:45:14.707939 13410 ts_tablet_manager.cc:1918] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:14.708163 13410 log.cc:1196] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/3f45c90500a2436ebd1f11ac8886a99d
I20260806 18:45:14.708549 13410 ts_tablet_manager.cc:1939] T 3f45c90500a2436ebd1f11ac8886a99d P 9a0dfc72c6fe460eb36872748f728337: Deleting consensus metadata
I20260806 18:45:14.709102 13408 ts_tablet_manager.cc:1918] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:14.709203 13408 log.cc:1196] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/3f45c90500a2436ebd1f11ac8886a99d
I20260806 18:45:14.709499 13408 ts_tablet_manager.cc:1939] T 3f45c90500a2436ebd1f11ac8886a99d P e583f83ff95447a0bd7a82de3b9a8e85: Deleting consensus metadata
I20260806 18:45:14.709930 12658 catalog_manager.cc:4841] TS 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933): tablet 3f45c90500a2436ebd1f11ac8886a99d (table test-table [id=de8eb48f10834e75b71aaf0434332b2b]) successfully deleted
I20260806 18:45:14.710582 12662 catalog_manager.cc:4841] TS e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729): tablet 3f45c90500a2436ebd1f11ac8886a99d (table test-table [id=de8eb48f10834e75b71aaf0434332b2b]) successfully deleted
I20260806 18:45:14.711377 13072 tablet_service.cc:1467] Processing CreateTablet for tablet 3de08d8464704f0e93b134a4cff9cadd (DEFAULT_TABLE table=test-table [id=96a096e98cd94d56bccf7220779b1fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:14.711382 12884 tablet_service.cc:1467] Processing CreateTablet for tablet 3de08d8464704f0e93b134a4cff9cadd (DEFAULT_TABLE table=test-table [id=96a096e98cd94d56bccf7220779b1fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:14.711549 13257 tablet_service.cc:1467] Processing CreateTablet for tablet 3de08d8464704f0e93b134a4cff9cadd (DEFAULT_TABLE table=test-table [id=96a096e98cd94d56bccf7220779b1fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:14.711639 12884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3de08d8464704f0e93b134a4cff9cadd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.711644 13072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3de08d8464704f0e93b134a4cff9cadd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.711763 13257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3de08d8464704f0e93b134a4cff9cadd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.713887 13380 tablet_bootstrap.cc:492] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: Bootstrap starting.
I20260806 18:45:14.713958 13378 tablet_bootstrap.cc:492] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: Bootstrap starting.
I20260806 18:45:14.714884 13378 tablet_bootstrap.cc:654] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.714893 13380 tablet_bootstrap.cc:654] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.715500 13377 tablet_bootstrap.cc:492] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: Bootstrap starting.
I20260806 18:45:14.716325 13378 tablet_bootstrap.cc:492] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: No bootstrap required, opened a new log
I20260806 18:45:14.716408 13378 ts_tablet_manager.cc:1397] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:14.716435 13377 tablet_bootstrap.cc:654] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.717113 13378 raft_consensus.cc:348] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.717221 13378 raft_consensus.cc:374] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.717252 13378 raft_consensus.cc:729] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0dfc72c6fe460eb36872748f728337, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.717378 13378 consensus_queue.cc:260] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.718428 13378 ts_tablet_manager.cc:1428] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:14.718462 13380 tablet_bootstrap.cc:492] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: No bootstrap required, opened a new log
I20260806 18:45:14.718544 13380 ts_tablet_manager.cc:1397] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260806 18:45:14.718995 13380 raft_consensus.cc:348] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.719111 13380 raft_consensus.cc:374] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.719147 13380 raft_consensus.cc:729] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e583f83ff95447a0bd7a82de3b9a8e85, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.719272 13380 consensus_queue.cc:260] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.719513 13380 ts_tablet_manager.cc:1428] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:45:14.720847 13377 tablet_bootstrap.cc:492] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: No bootstrap required, opened a new log
I20260806 18:45:14.720942 13377 ts_tablet_manager.cc:1397] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260806 18:45:14.721437 13377 raft_consensus.cc:348] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.721575 13377 raft_consensus.cc:374] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.721612 13377 raft_consensus.cc:729] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edff284cbe514cdb99e162e92d303825, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.721737 13377 consensus_queue.cc:260] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.721984 13377 ts_tablet_manager.cc:1428] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:14.900022 13404 raft_consensus.cc:482] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:14.900183 13404 raft_consensus.cc:504] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.900578 13404 leader_election.cc:288] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933), edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777)
I20260806 18:45:14.901134 12914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3de08d8464704f0e93b134a4cff9cadd" candidate_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0dfc72c6fe460eb36872748f728337" is_pre_election: true
I20260806 18:45:14.901142 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3de08d8464704f0e93b134a4cff9cadd" candidate_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff284cbe514cdb99e162e92d303825" is_pre_election: true
I20260806 18:45:14.901284 13091 raft_consensus.cc:2454] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e583f83ff95447a0bd7a82de3b9a8e85 in term 0.
I20260806 18:45:14.901294 12914 raft_consensus.cc:2454] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e583f83ff95447a0bd7a82de3b9a8e85 in term 0.
I20260806 18:45:14.901600 13196 leader_election.cc:302] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e583f83ff95447a0bd7a82de3b9a8e85, edff284cbe514cdb99e162e92d303825; no voters:
I20260806 18:45:14.901793 13404 raft_consensus.cc:2781] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:14.901851 13404 raft_consensus.cc:482] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:14.901880 13404 raft_consensus.cc:3037] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:14.903034 13404 raft_consensus.cc:504] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.903360 13404 leader_election.cc:288] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [CANDIDATE]: Term 1 election: Requested vote from peers 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933), edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777)
I20260806 18:45:14.903760 12914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3de08d8464704f0e93b134a4cff9cadd" candidate_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0dfc72c6fe460eb36872748f728337"
I20260806 18:45:14.903834 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3de08d8464704f0e93b134a4cff9cadd" candidate_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff284cbe514cdb99e162e92d303825"
I20260806 18:45:14.903883 12914 raft_consensus.cc:3037] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:14.903975 13091 raft_consensus.cc:3037] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:14.904811 12914 raft_consensus.cc:2454] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e583f83ff95447a0bd7a82de3b9a8e85 in term 1.
I20260806 18:45:14.904824 13091 raft_consensus.cc:2454] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e583f83ff95447a0bd7a82de3b9a8e85 in term 1.
I20260806 18:45:14.905079 13196 leader_election.cc:302] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a0dfc72c6fe460eb36872748f728337, e583f83ff95447a0bd7a82de3b9a8e85; no voters:
I20260806 18:45:14.905290 13404 raft_consensus.cc:2781] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:14.905404 13404 raft_consensus.cc:686] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 LEADER]: Becoming Leader. State: Replica: e583f83ff95447a0bd7a82de3b9a8e85, State: Running, Role: LEADER
I20260806 18:45:14.905556 13404 consensus_queue.cc:237] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.907501 12686 catalog_manager.cc:5495] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 reported cstate change: term changed from 0 to 1, leader changed from <none> to e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67). New cstate: current_term: 1 leader_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } health_report { overall_health: HEALTHY } } }
Aug 06 18:45:14 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041913, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.126@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:14.980253 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.972089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:41947 (local address 127.0.0.1:49920)
0806 18:45:14.972373 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:14.972404 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:14.972510 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:14.972726 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:14.972728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:14.973158 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:14.973260 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.973267 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.974010 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.974012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.974607 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:14.974616 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.975030 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.975032 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:14.975122 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.975612 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:14.975624 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:14.977035 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:14.978792 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.978801 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.978807 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.979033 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.979309 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:14.979313 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:14.979314 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:14.979405 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:14.979748 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:14.979753 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:14.979888 (+ 135us) client_negotiation.cc:769] Sending connection context
0806 18:45:14.979985 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 18:45:14.980103 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":14,"thread_start_us":108,"threads_started":1}
I20260806 18:45:14.980432 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:14.972139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49920 (local address 127.1.245.126:41947)
0806 18:45:14.972418 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:14.972425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:14.972456 (+ 31us) server_negotiation.cc:407] Connection header received
0806 18:45:14.972582 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:14.972585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:14.972626 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:14.972701 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:14.973374 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.973926 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.974714 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:14.974947 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:14.975021 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:14.977210 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:14.977219 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:14.977221 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:14.977244 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:14.978685 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.979110 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.979112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.979113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.979181 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:14.979485 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:14.979487 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:14.979488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:14.979646 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:14.979715 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:14.980083 (+ 368us) server_negotiation.cc:299] Negotiation successful
0806 18:45:14.980205 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":212,"thread_start_us":101,"threads_started":1}
I20260806 18:45:14.984393 12686 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:49920:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 18:45:14.985095 12686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:14.990299 13072 tablet_service.cc:1467] Processing CreateTablet for tablet 39b5b2a17c5441f08e18577f89f49ca0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77c374d764264704ae2b381e357467b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:45:14.990300 12884 tablet_service.cc:1467] Processing CreateTablet for tablet 39b5b2a17c5441f08e18577f89f49ca0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77c374d764264704ae2b381e357467b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:45:14.990581 13257 tablet_service.cc:1467] Processing CreateTablet for tablet 39b5b2a17c5441f08e18577f89f49ca0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=77c374d764264704ae2b381e357467b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 18:45:14.990775 13072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39b5b2a17c5441f08e18577f89f49ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.990775 12884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39b5b2a17c5441f08e18577f89f49ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.990938 13257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39b5b2a17c5441f08e18577f89f49ca0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:14.993162 13380 tablet_bootstrap.cc:492] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: Bootstrap starting.
I20260806 18:45:14.993705 13377 tablet_bootstrap.cc:492] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: Bootstrap starting.
I20260806 18:45:14.994071 13380 tablet_bootstrap.cc:654] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.994481 13378 tablet_bootstrap.cc:492] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: Bootstrap starting.
I20260806 18:45:14.994663 13377 tablet_bootstrap.cc:654] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.995335 13378 tablet_bootstrap.cc:654] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:14.995436 13380 tablet_bootstrap.cc:492] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: No bootstrap required, opened a new log
I20260806 18:45:14.995528 13380 ts_tablet_manager.cc:1397] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 18:45:14.995792 13377 tablet_bootstrap.cc:492] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: No bootstrap required, opened a new log
I20260806 18:45:14.995880 13377 ts_tablet_manager.cc:1397] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:14.995958 13380 raft_consensus.cc:348] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.996089 13380 raft_consensus.cc:374] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.996129 13380 raft_consensus.cc:729] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e583f83ff95447a0bd7a82de3b9a8e85, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.996284 13377 raft_consensus.cc:348] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.996259 13380 consensus_queue.cc:260] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.996398 13377 raft_consensus.cc:374] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.996438 13377 raft_consensus.cc:729] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edff284cbe514cdb99e162e92d303825, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.996588 13380 ts_tablet_manager.cc:1428] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260806 18:45:14.996556 13377 consensus_queue.cc:260] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.996803 13377 ts_tablet_manager.cc:1428] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:14.996788 13404 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:14.996922 13404 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:45:14.997020 13388 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:14.997133 13388 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:45:14.997926 13378 tablet_bootstrap.cc:492] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: No bootstrap required, opened a new log
I20260806 18:45:14.998036 13378 ts_tablet_manager.cc:1397] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 18:45:14.998508 13378 raft_consensus.cc:348] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.998628 13378 raft_consensus.cc:374] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:14.998669 13378 raft_consensus.cc:729] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0dfc72c6fe460eb36872748f728337, State: Initialized, Role: FOLLOWER
I20260806 18:45:14.998799 13378 consensus_queue.cc:260] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:14.999066 13378 ts_tablet_manager.cc:1428] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260806 18:45:14.999261 13387 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:14.999379 13387 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:45:15.279811 13388 raft_consensus.cc:482] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:15.279970 13388 raft_consensus.cc:504] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.281159 13388 leader_election.cc:288] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933), e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729)
I20260806 18:45:15.285188 13430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.281493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:38729 (local address 127.1.245.66:44745)
0806 18:45:15.281849 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.281864 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.281966 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.282216 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.282219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.282262 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:15.282402 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.282408 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.283138 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.283141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.284002 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.284012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.284732 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.284736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.284843 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.284868 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.284942 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.285013 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"mutex_wait_us":22,"thread_start_us":70,"threads_started":1}
I20260806 18:45:15.285192 13432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.281441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:38933 (local address 127.1.245.66:37953)
0806 18:45:15.281783 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.281810 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.281933 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.282178 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.282181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.282196 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:15.282363 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.282369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.283078 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.283082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.284039 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.284049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.284863 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.284867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.284940 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.284959 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.285014 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.285066 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":62,"spinlock_wait_cycles":81152,"thread_start_us":97,"threads_started":1}
I20260806 18:45:15.285223 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.281575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:44745 (local address 127.1.245.67:38729)
0806 18:45:15.281919 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.281923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.281939 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:45:15.282075 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.282078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.282121 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.282192 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:15.282511 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.283031 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.284111 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.284617 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.284731 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.284792 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.284980 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.285072 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":90,"threads_started":1}
I20260806 18:45:15.285233 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.281554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:37953 (local address 127.1.245.65:38933)
0806 18:45:15.281786 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.281791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.281890 (+ 99us) server_negotiation.cc:407] Connection header received
0806 18:45:15.282037 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.282039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.282080 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.282151 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:15.282468 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.282973 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.284166 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.284678 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.284798 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.284853 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.285043 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.285119 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":57,"threads_started":1}
I20260806 18:45:15.285897 12914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39b5b2a17c5441f08e18577f89f49ca0" candidate_uuid: "edff284cbe514cdb99e162e92d303825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0dfc72c6fe460eb36872748f728337" is_pre_election: true
I20260806 18:45:15.285980 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39b5b2a17c5441f08e18577f89f49ca0" candidate_uuid: "edff284cbe514cdb99e162e92d303825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" is_pre_election: true
I20260806 18:45:15.286070 12914 raft_consensus.cc:2454] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edff284cbe514cdb99e162e92d303825 in term 0.
I20260806 18:45:15.286224 13277 raft_consensus.cc:2454] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate edff284cbe514cdb99e162e92d303825 in term 0.
I20260806 18:45:15.286497 13009 leader_election.cc:302] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a0dfc72c6fe460eb36872748f728337, edff284cbe514cdb99e162e92d303825; no voters:
I20260806 18:45:15.286826 13388 raft_consensus.cc:2781] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:15.286885 13388 raft_consensus.cc:482] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:15.286911 13388 raft_consensus.cc:3037] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.287884 13388 raft_consensus.cc:504] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.288229 13388 leader_election.cc:288] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [CANDIDATE]: Term 1 election: Requested vote from peers 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933), e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729)
I20260806 18:45:15.288604 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39b5b2a17c5441f08e18577f89f49ca0" candidate_uuid: "edff284cbe514cdb99e162e92d303825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e583f83ff95447a0bd7a82de3b9a8e85"
I20260806 18:45:15.288619 12914 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39b5b2a17c5441f08e18577f89f49ca0" candidate_uuid: "edff284cbe514cdb99e162e92d303825" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a0dfc72c6fe460eb36872748f728337"
I20260806 18:45:15.288733 13277 raft_consensus.cc:3037] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.288745 12914 raft_consensus.cc:3037] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.289742 13277 raft_consensus.cc:2454] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edff284cbe514cdb99e162e92d303825 in term 1.
I20260806 18:45:15.289757 12914 raft_consensus.cc:2454] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate edff284cbe514cdb99e162e92d303825 in term 1.
I20260806 18:45:15.290046 13012 leader_election.cc:302] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9a0dfc72c6fe460eb36872748f728337, e583f83ff95447a0bd7a82de3b9a8e85, edff284cbe514cdb99e162e92d303825; no voters:
I20260806 18:45:15.290218 13388 raft_consensus.cc:2781] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:15.290427 13388 raft_consensus.cc:686] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [term 1 LEADER]: Becoming Leader. State: Replica: edff284cbe514cdb99e162e92d303825, State: Running, Role: LEADER
I20260806 18:45:15.290668 13388 consensus_queue.cc:237] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.291898 13437 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: TxnStatusTablet state changed. Reason: New leader edff284cbe514cdb99e162e92d303825. Latest consensus state: current_term: 1 leader_uuid: "edff284cbe514cdb99e162e92d303825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:15.292018 13437 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:45:15.292303 13440 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 18:45:15.292464 13440 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 18:45:15.293262 12687 catalog_manager.cc:5495] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 reported cstate change: term changed from 0 to 1, leader changed from <none> to edff284cbe514cdb99e162e92d303825 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "edff284cbe514cdb99e162e92d303825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } health_report { overall_health: UNKNOWN } } }
I20260806 18:45:15.366410 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.361633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:42777 (local address 127.0.0.1:48616)
0806 18:45:15.361798 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.361815 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.361931 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.362447 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.362451 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.362460 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:45:15.362583 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.362590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.363522 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.363526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.364515 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.364528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.365025 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.365029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.365126 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.365190 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.365968 (+ 778us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.365982 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.366109 (+ 127us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.366212 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:45:15.366430 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.361741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48616 (local address 127.1.245.66:42777)
0806 18:45:15.362137 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.362143 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.362154 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:45:15.362219 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.362221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.362279 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.362408 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:15.362743 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.363361 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.364669 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.364913 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.364997 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.365387 (+ 390us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.365870 (+ 483us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.365978 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.366153 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.366222 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":114,"threads_started":1}
I20260806 18:45:15.369642 12914 raft_consensus.cc:1261] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Refusing update from remote peer edff284cbe514cdb99e162e92d303825: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:15.369810 13277 raft_consensus.cc:1261] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Refusing update from remote peer edff284cbe514cdb99e162e92d303825: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:15.370208 13437 consensus_queue.cc:1035] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:15.370401 13388 consensus_queue.cc:1035] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:15.371850 13387 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: TxnStatusTablet state changed. Reason: New leader edff284cbe514cdb99e162e92d303825. Latest consensus state: current_term: 1 leader_uuid: "edff284cbe514cdb99e162e92d303825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:15.371975 13387 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:45:15.373206 13404 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: TxnStatusTablet state changed. Reason: New leader edff284cbe514cdb99e162e92d303825. Latest consensus state: current_term: 1 leader_uuid: "edff284cbe514cdb99e162e92d303825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:15.373334 13404 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:45:15.373746 13388 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "edff284cbe514cdb99e162e92d303825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:15.373878 13388 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:45:15.374346 13387 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "edff284cbe514cdb99e162e92d303825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:15.374450 13387 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P 9a0dfc72c6fe460eb36872748f728337: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:45:15.376386 13437 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "edff284cbe514cdb99e162e92d303825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:15.376513 13437 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P edff284cbe514cdb99e162e92d303825: This TxnStatusTablet replica's current role is: LEADER
I20260806 18:45:15.377447 13404 tablet_replica.cc:439] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "edff284cbe514cdb99e162e92d303825" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } } }
I20260806 18:45:15.377580 13404 tablet_replica.cc:442] T 39b5b2a17c5441f08e18577f89f49ca0 P e583f83ff95447a0bd7a82de3b9a8e85: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 18:45:15.377791 13450 mvcc.cc:205] Tried to move back new op lower bound from 7315627685348274176 to 7315627685033193472. Current Snapshot: MvccSnapshot[applied={T|T < 7315627685348274176}]
I20260806 18:45:15.391280 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.387899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:35007 (local address 127.1.245.66:42777)
0806 18:45:15.388020 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.388024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.388237 (+ 213us) server_negotiation.cc:407] Connection header received
0806 18:45:15.388388 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.388391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.388447 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.388507 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:15.388732 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.389238 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.390188 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.390475 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.390566 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.390852 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.390933 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.390988 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.391077 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.391138 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:45:15.391287 13459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.387832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:42777 (local address 127.1.245.67:35007)
0806 18:45:15.388155 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.388168 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.388277 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.388526 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.388528 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.388536 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:45:15.388639 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.388644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.389338 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.389341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.390074 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.390085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.390582 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.390585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.390675 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.390720 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.391003 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.391010 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.391067 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.391159 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":138,"threads_started":1}
I20260806 18:45:15.399093 13091 raft_consensus.cc:1261] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Refusing update from remote peer e583f83ff95447a0bd7a82de3b9a8e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:15.399122 12914 raft_consensus.cc:1261] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Refusing update from remote peer e583f83ff95447a0bd7a82de3b9a8e85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:15.399551 13404 consensus_queue.cc:1035] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:15.399729 13391 consensus_queue.cc:1035] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:15.425746 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.421658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:38729 (local address 127.1.245.66:51059)
0806 18:45:15.422060 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.422076 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.422188 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.422530 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.422533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.422543 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:45:15.422683 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.422690 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.423554 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.423557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.424474 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.424484 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.425064 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.425068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.425141 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.425192 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.425478 (+ 286us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.425482 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.425525 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.425593 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":103,"threads_started":1}
I20260806 18:45:15.425731 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.421745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:51059 (local address 127.1.245.67:38729)
0806 18:45:15.421875 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.421880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.422142 (+ 262us) server_negotiation.cc:407] Connection header received
0806 18:45:15.422304 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.422307 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.422422 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.422503 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:15.422796 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.423432 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.424596 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.424918 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.425027 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.425314 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.425403 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.425478 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.425539 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.425596 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:45:15.458568 12686 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:15.458727 12686 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:15.460955 12686 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: Sending DeleteTablet for 3 replicas of tablet 3de08d8464704f0e93b134a4cff9cadd
I20260806 18:45:15.461568 12884 tablet_service.cc:1514] Processing DeleteTablet for tablet 3de08d8464704f0e93b134a4cff9cadd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:57206
I20260806 18:45:15.461583 13072 tablet_service.cc:1514] Processing DeleteTablet for tablet 3de08d8464704f0e93b134a4cff9cadd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:48600
I20260806 18:45:15.461709 13257 tablet_service.cc:1514] Processing DeleteTablet for tablet 3de08d8464704f0e93b134a4cff9cadd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:40474
I20260806 18:45:15.461995 13468 tablet_replica.cc:330] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: stopping tablet replica
I20260806 18:45:15.462004 13473 tablet_replica.cc:330] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: stopping tablet replica
I20260806 18:45:15.462127 13473 raft_consensus.cc:2229] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:15.462128 13468 raft_consensus.cc:2229] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:15.462252 13468 raft_consensus.cc:2258] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.462415 13469 tablet_replica.cc:330] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: stopping tablet replica
I20260806 18:45:15.462555 13469 raft_consensus.cc:2229] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:15.462373 13473 raft_consensus.cc:2258] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.462702 13469 raft_consensus.cc:2258] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.463167 13468 ts_tablet_manager.cc:1905] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:15.463793 13469 ts_tablet_manager.cc:1905] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:15.465987 13468 ts_tablet_manager.cc:1918] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:45:15.466069 13468 log.cc:1196] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/3de08d8464704f0e93b134a4cff9cadd
I20260806 18:45:15.466291 13468 ts_tablet_manager.cc:1939] T 3de08d8464704f0e93b134a4cff9cadd P edff284cbe514cdb99e162e92d303825: Deleting consensus metadata
I20260806 18:45:15.466259 13469 ts_tablet_manager.cc:1918] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:45:15.466388 13469 log.cc:1196] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/3de08d8464704f0e93b134a4cff9cadd
I20260806 18:45:15.466638 13469 ts_tablet_manager.cc:1939] T 3de08d8464704f0e93b134a4cff9cadd P 9a0dfc72c6fe460eb36872748f728337: Deleting consensus metadata
I20260806 18:45:15.467008 13473 ts_tablet_manager.cc:1905] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:15.467103 12658 catalog_manager.cc:4841] TS edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777): tablet 3de08d8464704f0e93b134a4cff9cadd (table test-table [id=96a096e98cd94d56bccf7220779b1fe6]) successfully deleted
I20260806 18:45:15.467681 12658 catalog_manager.cc:4841] TS 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933): tablet 3de08d8464704f0e93b134a4cff9cadd (table test-table [id=96a096e98cd94d56bccf7220779b1fe6]) successfully deleted
I20260806 18:45:15.469817 13473 ts_tablet_manager.cc:1918] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:45:15.469931 13473 log.cc:1196] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/3de08d8464704f0e93b134a4cff9cadd
I20260806 18:45:15.470224 13473 ts_tablet_manager.cc:1939] T 3de08d8464704f0e93b134a4cff9cadd P e583f83ff95447a0bd7a82de3b9a8e85: Deleting consensus metadata
I20260806 18:45:15.471172 12662 catalog_manager.cc:4841] TS e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729): tablet 3de08d8464704f0e93b134a4cff9cadd (table test-table [id=96a096e98cd94d56bccf7220779b1fe6]) successfully deleted
Aug 06 18:45:15 dist-test-slave-fgb3 krb5kdc[12621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 18:45:15 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:15 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:45:15.488714 13489 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:15.481856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49930 (local address 127.1.245.126:41947)
0806 18:45:15.482118 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.482123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.482211 (+ 88us) server_negotiation.cc:407] Connection header received
0806 18:45:15.482449 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.482452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.482501 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.482581 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:15.483340 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.483909 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.484789 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.485037 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.485131 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.488557 (+ 3426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49930: BlockingRecv error: recv got EOF from 127.0.0.1:49930 (error 108)
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":80,"threads_started":1}
Aug 06 18:45:15 dist-test-slave-fgb3 krb5kdc[12621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041915, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM
I20260806 18:45:15.498394 13489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.490229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.1.245.126:41947)
0806 18:45:15.490425 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.490428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.490442 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:15.490551 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.490553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.490596 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.490655 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:15.491270 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.491839 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.492630 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.492885 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.492980 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.495387 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:15.495390 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:15.495392 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:15.495418 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:15.496722 (+ 1304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:15.497153 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:15.497157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:15.497158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:15.497268 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:15.497558 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:15.497560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:15.497561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:15.497738 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:15.497846 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.498056 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.498157 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:45:15.500841 12686 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:45:15.501281 12686 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:15.505894 12884 tablet_service.cc:1467] Processing CreateTablet for tablet 25a18592829a40e8b1768e61a58cba3d (DEFAULT_TABLE table=test-table [id=e5de3e78fe704e79b4e3c087af817324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:15.505896 13072 tablet_service.cc:1467] Processing CreateTablet for tablet 25a18592829a40e8b1768e61a58cba3d (DEFAULT_TABLE table=test-table [id=e5de3e78fe704e79b4e3c087af817324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:15.505977 13257 tablet_service.cc:1467] Processing CreateTablet for tablet 25a18592829a40e8b1768e61a58cba3d (DEFAULT_TABLE table=test-table [id=e5de3e78fe704e79b4e3c087af817324]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:15.506233 13072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25a18592829a40e8b1768e61a58cba3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:15.506233 13257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25a18592829a40e8b1768e61a58cba3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:15.506245 12884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25a18592829a40e8b1768e61a58cba3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:15.508836 13498 tablet_bootstrap.cc:492] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: Bootstrap starting.
I20260806 18:45:15.508839 13501 tablet_bootstrap.cc:492] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: Bootstrap starting.
I20260806 18:45:15.509065 13500 tablet_bootstrap.cc:492] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: Bootstrap starting.
I20260806 18:45:15.509685 13501 tablet_bootstrap.cc:654] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:15.509817 13498 tablet_bootstrap.cc:654] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:15.510022 13500 tablet_bootstrap.cc:654] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:15.510776 13501 tablet_bootstrap.cc:492] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: No bootstrap required, opened a new log
I20260806 18:45:15.510875 13501 ts_tablet_manager.cc:1397] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:15.511076 13500 tablet_bootstrap.cc:492] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: No bootstrap required, opened a new log
I20260806 18:45:15.511162 13500 ts_tablet_manager.cc:1397] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:15.511163 13498 tablet_bootstrap.cc:492] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: No bootstrap required, opened a new log
I20260806 18:45:15.511310 13501 raft_consensus.cc:348] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.511394 13498 ts_tablet_manager.cc:1397] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260806 18:45:15.511427 13501 raft_consensus.cc:374] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:15.511473 13501 raft_consensus.cc:729] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edff284cbe514cdb99e162e92d303825, State: Initialized, Role: FOLLOWER
I20260806 18:45:15.511536 13500 raft_consensus.cc:348] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.511631 13500 raft_consensus.cc:374] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:15.511660 13500 raft_consensus.cc:729] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e583f83ff95447a0bd7a82de3b9a8e85, State: Initialized, Role: FOLLOWER
I20260806 18:45:15.511629 13501 consensus_queue.cc:260] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.511802 13500 consensus_queue.cc:260] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.511866 13498 raft_consensus.cc:348] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.511976 13498 raft_consensus.cc:374] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:15.512010 13498 raft_consensus.cc:729] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0dfc72c6fe460eb36872748f728337, State: Initialized, Role: FOLLOWER
I20260806 18:45:15.512122 13498 consensus_queue.cc:260] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.512377 13498 ts_tablet_manager.cc:1428] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:45:15.512620 13501 ts_tablet_manager.cc:1428] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:15.512653 13500 ts_tablet_manager.cc:1428] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:15.534876 13387 raft_consensus.cc:482] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:15.535038 13387 raft_consensus.cc:504] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.536216 13387 leader_election.cc:288] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777), e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729)
I20260806 18:45:15.540923 13511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.536591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:38729 (local address 127.1.245.65:51123)
0806 18:45:15.536966 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.536979 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.537079 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.537450 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.537452 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.537468 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:15.537577 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.537582 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.538534 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.538538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.539646 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.539656 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.540478 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.540482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.540587 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.540617 (+ 30us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.540690 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.540766 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"spinlock_wait_cycles":768,"thread_start_us":121,"threads_started":1}
I20260806 18:45:15.540915 13435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.537033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:51123 (local address 127.1.245.67:38729)
0806 18:45:15.537170 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.537174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.537188 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:15.537240 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.537243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.537287 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.537355 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:15.537686 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.538424 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.539796 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.540311 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.540417 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.540462 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.540711 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.540783 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:45:15.541185 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.536671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:42757 (local address 127.1.245.66:42777)
0806 18:45:15.536804 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.536809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.536999 (+ 190us) server_negotiation.cc:407] Connection header received
0806 18:45:15.537187 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.537191 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.537252 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.537294 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:15.538169 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.538874 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.540110 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.540598 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.540725 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.540773 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.540984 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.541076 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:45:15.541177 13507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.536538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:42777 (local address 127.1.245.65:42757)
0806 18:45:15.536888 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.536903 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.537037 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.537448 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.537450 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.537484 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:15.538027 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.538038 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.539012 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.539015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.539964 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.539976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.540746 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.540750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.540845 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.540870 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.540949 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.541024 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":43,"spinlock_wait_cycles":768,"thread_start_us":98,"threads_started":1}
I20260806 18:45:15.541831 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25a18592829a40e8b1768e61a58cba3d" candidate_uuid: "9a0dfc72c6fe460eb36872748f728337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" is_pre_election: true
I20260806 18:45:15.541951 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25a18592829a40e8b1768e61a58cba3d" candidate_uuid: "9a0dfc72c6fe460eb36872748f728337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff284cbe514cdb99e162e92d303825" is_pre_election: true
I20260806 18:45:15.542011 13277 raft_consensus.cc:2454] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0dfc72c6fe460eb36872748f728337 in term 0.
I20260806 18:45:15.542092 13091 raft_consensus.cc:2454] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0dfc72c6fe460eb36872748f728337 in term 0.
I20260806 18:45:15.542423 12822 leader_election.cc:302] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9a0dfc72c6fe460eb36872748f728337, e583f83ff95447a0bd7a82de3b9a8e85, edff284cbe514cdb99e162e92d303825; no voters:
I20260806 18:45:15.542691 13387 raft_consensus.cc:2781] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:15.542755 13387 raft_consensus.cc:482] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:15.542800 13387 raft_consensus.cc:3037] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.543813 13387 raft_consensus.cc:504] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.544142 13387 leader_election.cc:288] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [CANDIDATE]: Term 1 election: Requested vote from peers edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777), e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729)
I20260806 18:45:15.544535 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25a18592829a40e8b1768e61a58cba3d" candidate_uuid: "9a0dfc72c6fe460eb36872748f728337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff284cbe514cdb99e162e92d303825"
I20260806 18:45:15.544535 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25a18592829a40e8b1768e61a58cba3d" candidate_uuid: "9a0dfc72c6fe460eb36872748f728337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e583f83ff95447a0bd7a82de3b9a8e85"
I20260806 18:45:15.544668 13277 raft_consensus.cc:3037] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.544668 13091 raft_consensus.cc:3037] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.545820 13277 raft_consensus.cc:2454] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0dfc72c6fe460eb36872748f728337 in term 1.
I20260806 18:45:15.545820 13091 raft_consensus.cc:2454] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0dfc72c6fe460eb36872748f728337 in term 1.
I20260806 18:45:15.546128 12816 leader_election.cc:302] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a0dfc72c6fe460eb36872748f728337, edff284cbe514cdb99e162e92d303825; no voters:
I20260806 18:45:15.546411 13387 raft_consensus.cc:2781] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:15.546638 13387 raft_consensus.cc:686] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 1 LEADER]: Becoming Leader. State: Replica: 9a0dfc72c6fe460eb36872748f728337, State: Running, Role: LEADER
I20260806 18:45:15.546896 13387 consensus_queue.cc:237] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.549168 12687 catalog_manager.cc:5495] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65). New cstate: current_term: 1 leader_uuid: "9a0dfc72c6fe460eb36872748f728337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } health_report { overall_health: UNKNOWN } } }
I20260806 18:45:15.563930 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.560561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.1.245.65:38933)
0806 18:45:15.560701 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.560705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.560723 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:45:15.560811 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.560813 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.560877 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.561008 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:15.561188 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.561723 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.562573 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.562795 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.562879 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.563083 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.563588 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.563677 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.563725 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.563784 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:45:15.566880 13277 raft_consensus.cc:1261] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Refusing update from remote peer 9a0dfc72c6fe460eb36872748f728337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:15.566892 13091 raft_consensus.cc:1261] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Refusing update from remote peer 9a0dfc72c6fe460eb36872748f728337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:15.567414 13513 consensus_queue.cc:1035] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:15.567682 13387 consensus_queue.cc:1035] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:15.578670 12687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:15.578851 12687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:15.581009 12687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: Sending DeleteTablet for 3 replicas of tablet 25a18592829a40e8b1768e61a58cba3d
I20260806 18:45:15.581672 13072 tablet_service.cc:1514] Processing DeleteTablet for tablet 25a18592829a40e8b1768e61a58cba3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:48600
I20260806 18:45:15.581719 12884 tablet_service.cc:1514] Processing DeleteTablet for tablet 25a18592829a40e8b1768e61a58cba3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:57206
I20260806 18:45:15.581869 13468 tablet_replica.cc:330] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: stopping tablet replica
I20260806 18:45:15.581908 13469 tablet_replica.cc:330] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: stopping tablet replica
I20260806 18:45:15.581996 13468 raft_consensus.cc:2229] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:15.582015 13469 raft_consensus.cc:2229] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:15.582146 13468 raft_consensus.cc:2258] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.582360 13469 raft_consensus.cc:2258] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.582535 13257 tablet_service.cc:1514] Processing DeleteTablet for tablet 25a18592829a40e8b1768e61a58cba3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:40474
I20260806 18:45:15.582705 13473 tablet_replica.cc:330] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: stopping tablet replica
I20260806 18:45:15.582823 13473 raft_consensus.cc:2229] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:15.582952 13473 raft_consensus.cc:2258] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.583009 12687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 18:45:15.583114 13468 ts_tablet_manager.cc:1905] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:45:15.583433 12687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:15.585770 13468 ts_tablet_manager.cc:1918] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:15.586630 13469 ts_tablet_manager.cc:1905] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:15.588194 12884 tablet_service.cc:1467] Processing CreateTablet for tablet f388d82bbe8a44b6a82dba34b498b3de (DEFAULT_TABLE table=test-table [id=742f53f08ca04c4a90217375bee021d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:15.588485 12884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f388d82bbe8a44b6a82dba34b498b3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:15.588685 13473 ts_tablet_manager.cc:1905] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:15.589068 13469 ts_tablet_manager.cc:1918] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:15.589135 13469 log.cc:1196] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/25a18592829a40e8b1768e61a58cba3d
I20260806 18:45:15.589365 13469 ts_tablet_manager.cc:1939] T 25a18592829a40e8b1768e61a58cba3d P 9a0dfc72c6fe460eb36872748f728337: Deleting consensus metadata
I20260806 18:45:15.590771 13257 tablet_service.cc:1467] Processing CreateTablet for tablet f388d82bbe8a44b6a82dba34b498b3de (DEFAULT_TABLE table=test-table [id=742f53f08ca04c4a90217375bee021d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:15.591030 13257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f388d82bbe8a44b6a82dba34b498b3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:15.591327 13473 ts_tablet_manager.cc:1918] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:15.591393 13473 log.cc:1196] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/25a18592829a40e8b1768e61a58cba3d
I20260806 18:45:15.591609 13473 ts_tablet_manager.cc:1939] T 25a18592829a40e8b1768e61a58cba3d P e583f83ff95447a0bd7a82de3b9a8e85: Deleting consensus metadata
I20260806 18:45:15.593081 13500 tablet_bootstrap.cc:492] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: Bootstrap starting.
I20260806 18:45:15.593163 13498 tablet_bootstrap.cc:492] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337: Bootstrap starting.
I20260806 18:45:15.593813 13500 tablet_bootstrap.cc:654] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:15.594120 13498 tablet_bootstrap.cc:654] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:15.594874 13500 tablet_bootstrap.cc:492] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: No bootstrap required, opened a new log
I20260806 18:45:15.594962 13500 ts_tablet_manager.cc:1397] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:15.595038 12658 catalog_manager.cc:4841] TS 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933): tablet 25a18592829a40e8b1768e61a58cba3d (table test-table [id=e5de3e78fe704e79b4e3c087af817324]) successfully deleted
I20260806 18:45:15.595273 12662 catalog_manager.cc:4841] TS e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729): tablet 25a18592829a40e8b1768e61a58cba3d (table test-table [id=e5de3e78fe704e79b4e3c087af817324]) successfully deleted
I20260806 18:45:15.595350 13500 raft_consensus.cc:348] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.585852 13468 log.cc:1196] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/25a18592829a40e8b1768e61a58cba3d
I20260806 18:45:15.595463 13500 raft_consensus.cc:374] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:15.595495 13500 raft_consensus.cc:729] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e583f83ff95447a0bd7a82de3b9a8e85, State: Initialized, Role: FOLLOWER
I20260806 18:45:15.590394 13072 tablet_service.cc:1467] Processing CreateTablet for tablet f388d82bbe8a44b6a82dba34b498b3de (DEFAULT_TABLE table=test-table [id=742f53f08ca04c4a90217375bee021d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:15.595610 13500 consensus_queue.cc:260] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.595739 13468 ts_tablet_manager.cc:1939] T 25a18592829a40e8b1768e61a58cba3d P edff284cbe514cdb99e162e92d303825: Deleting consensus metadata
I20260806 18:45:15.595750 13072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f388d82bbe8a44b6a82dba34b498b3de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:15.595865 13500 ts_tablet_manager.cc:1428] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:15.596448 12658 catalog_manager.cc:4841] TS edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777): tablet 25a18592829a40e8b1768e61a58cba3d (table test-table [id=e5de3e78fe704e79b4e3c087af817324]) successfully deleted
I20260806 18:45:15.597065 13498 tablet_bootstrap.cc:492] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337: No bootstrap required, opened a new log
I20260806 18:45:15.597146 13498 ts_tablet_manager.cc:1397] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260806 18:45:15.597599 13498 raft_consensus.cc:348] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.597724 13498 raft_consensus.cc:374] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:15.597759 13498 raft_consensus.cc:729] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a0dfc72c6fe460eb36872748f728337, State: Initialized, Role: FOLLOWER
I20260806 18:45:15.597898 13498 consensus_queue.cc:260] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.598084 13501 tablet_bootstrap.cc:492] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: Bootstrap starting.
I20260806 18:45:15.598191 13498 ts_tablet_manager.cc:1428] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:45:15.598994 13501 tablet_bootstrap.cc:654] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:15.600916 13501 tablet_bootstrap.cc:492] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: No bootstrap required, opened a new log
I20260806 18:45:15.601001 13501 ts_tablet_manager.cc:1397] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 18:45:15.601351 13501 raft_consensus.cc:348] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.601441 13501 raft_consensus.cc:374] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:15.601472 13501 raft_consensus.cc:729] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edff284cbe514cdb99e162e92d303825, State: Initialized, Role: FOLLOWER
I20260806 18:45:15.601586 13501 consensus_queue.cc:260] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.601809 13501 ts_tablet_manager.cc:1428] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:45:15.776597 13513 raft_consensus.cc:482] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:15.776758 13513 raft_consensus.cc:504] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.777267 13513 leader_election.cc:288] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777), e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729)
I20260806 18:45:15.777812 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f388d82bbe8a44b6a82dba34b498b3de" candidate_uuid: "9a0dfc72c6fe460eb36872748f728337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" is_pre_election: true
I20260806 18:45:15.777814 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f388d82bbe8a44b6a82dba34b498b3de" candidate_uuid: "9a0dfc72c6fe460eb36872748f728337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff284cbe514cdb99e162e92d303825" is_pre_election: true
I20260806 18:45:15.777964 13277 raft_consensus.cc:2454] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0dfc72c6fe460eb36872748f728337 in term 0.
I20260806 18:45:15.778005 13091 raft_consensus.cc:2454] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a0dfc72c6fe460eb36872748f728337 in term 0.
I20260806 18:45:15.778302 12822 leader_election.cc:302] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a0dfc72c6fe460eb36872748f728337, e583f83ff95447a0bd7a82de3b9a8e85; no voters:
I20260806 18:45:15.778577 13513 raft_consensus.cc:2781] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:15.778646 13513 raft_consensus.cc:482] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:15.778676 13513 raft_consensus.cc:3037] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.779701 13513 raft_consensus.cc:504] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.780046 13513 leader_election.cc:288] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [CANDIDATE]: Term 1 election: Requested vote from peers edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777), e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729)
I20260806 18:45:15.780458 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f388d82bbe8a44b6a82dba34b498b3de" candidate_uuid: "9a0dfc72c6fe460eb36872748f728337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edff284cbe514cdb99e162e92d303825"
I20260806 18:45:15.780504 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f388d82bbe8a44b6a82dba34b498b3de" candidate_uuid: "9a0dfc72c6fe460eb36872748f728337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e583f83ff95447a0bd7a82de3b9a8e85"
I20260806 18:45:15.780598 13091 raft_consensus.cc:3037] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.780611 13277 raft_consensus.cc:3037] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:15.781728 13277 raft_consensus.cc:2454] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0dfc72c6fe460eb36872748f728337 in term 1.
I20260806 18:45:15.781728 13091 raft_consensus.cc:2454] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a0dfc72c6fe460eb36872748f728337 in term 1.
I20260806 18:45:15.782030 12816 leader_election.cc:302] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a0dfc72c6fe460eb36872748f728337, edff284cbe514cdb99e162e92d303825; no voters:
I20260806 18:45:15.782217 13513 raft_consensus.cc:2781] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:15.782372 13513 raft_consensus.cc:686] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 1 LEADER]: Becoming Leader. State: Replica: 9a0dfc72c6fe460eb36872748f728337, State: Running, Role: LEADER
I20260806 18:45:15.782531 13513 consensus_queue.cc:237] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } }
I20260806 18:45:15.784241 12687 catalog_manager.cc:5495] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65). New cstate: current_term: 1 leader_uuid: "9a0dfc72c6fe460eb36872748f728337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a0dfc72c6fe460eb36872748f728337" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 38933 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 } health_report { overall_health: UNKNOWN } } }
I20260806 18:45:15.849377 13529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.846004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:42777 (local address 127.1.245.65:34945)
0806 18:45:15.846372 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.846386 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.846497 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.846730 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.846733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.846742 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:45:15.846865 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.846871 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.847584 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.847588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.848219 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.848236 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.848658 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.848662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.848752 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.848800 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.849083 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.849091 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.849150 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.849252 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":87,"threads_started":1}
I20260806 18:45:15.849363 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.846128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:34945 (local address 127.1.245.66:42777)
0806 18:45:15.846361 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.846366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.846458 (+ 92us) server_negotiation.cc:407] Connection header received
0806 18:45:15.846598 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.846601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.846649 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.846711 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:15.846966 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.847482 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.848345 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.848560 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.848646 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.848933 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.849010 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.849069 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.849159 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.849215 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 18:45:15.855022 13091 raft_consensus.cc:1261] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Refusing update from remote peer 9a0dfc72c6fe460eb36872748f728337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:15.855021 13277 raft_consensus.cc:1261] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Refusing update from remote peer 9a0dfc72c6fe460eb36872748f728337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:15.855448 13387 consensus_queue.cc:1035] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edff284cbe514cdb99e162e92d303825" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:15.855607 13513 consensus_queue.cc:1035] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e583f83ff95447a0bd7a82de3b9a8e85" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 38729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:15.872931 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.868902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:38933 (local address 127.1.245.66:37117)
0806 18:45:15.869150 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.869166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:15.869249 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:15.869600 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:15.869603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:15.869611 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 18:45:15.869707 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.869713 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.870689 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.870693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.871641 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:15.871650 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.872204 (+ 554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.872208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:15.872299 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.872333 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.872631 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.872637 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:45:15.872696 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 18:45:15.872784 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260806 18:45:15.873574 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.869128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:37117 (local address 127.1.245.65:38933)
0806 18:45:15.869227 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:15.869231 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:15.869244 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:15.869369 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:15.869373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:15.869425 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:15.869487 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:15.869859 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.870588 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.871772 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:15.872083 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:15.872181 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:15.872463 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:15.872553 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:15.872621 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:15.873388 (+ 767us) server_negotiation.cc:299] Negotiation successful
0806 18:45:15.873456 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:45:15.894954 12682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:15.895105 12682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:15.897071 12682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d2eccca0d0d64b40aa4bf7c4e3cdf553: Sending DeleteTablet for 3 replicas of tablet f388d82bbe8a44b6a82dba34b498b3de
I20260806 18:45:15.897756 12884 tablet_service.cc:1514] Processing DeleteTablet for tablet f388d82bbe8a44b6a82dba34b498b3de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:57206
I20260806 18:45:15.897776 13072 tablet_service.cc:1514] Processing DeleteTablet for tablet f388d82bbe8a44b6a82dba34b498b3de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:48600
I20260806 18:45:15.897779 13257 tablet_service.cc:1514] Processing DeleteTablet for tablet f388d82bbe8a44b6a82dba34b498b3de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:15 UTC) from {username='oryx', principal='oryx/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:40474
I20260806 18:45:15.897945 13469 tablet_replica.cc:330] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337: stopping tablet replica
I20260806 18:45:15.897992 13473 tablet_replica.cc:330] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: stopping tablet replica
I20260806 18:45:15.898092 13473 raft_consensus.cc:2229] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:15.898092 13469 raft_consensus.cc:2229] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:15.898185 13473 raft_consensus.cc:2258] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.898281 13469 raft_consensus.cc:2258] T f388d82bbe8a44b6a82dba34b498b3de P 9a0dfc72c6fe460eb36872748f728337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.898478 13468 tablet_replica.cc:330] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: stopping tablet replica
I20260806 18:45:15.898602 13468 raft_consensus.cc:2229] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:15.898710 13468 raft_consensus.cc:2258] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:15.898741 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 12790
I20260806 18:45:15.898909 13473 ts_tablet_manager.cc:1905] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:15.901212 13473 ts_tablet_manager.cc:1918] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:45:15.901275 13473 log.cc:1196] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/f388d82bbe8a44b6a82dba34b498b3de
I20260806 18:45:15.901474 13473 ts_tablet_manager.cc:1939] T f388d82bbe8a44b6a82dba34b498b3de P e583f83ff95447a0bd7a82de3b9a8e85: Deleting consensus metadata
I20260806 18:45:15.907096 13468 ts_tablet_manager.cc:1905] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:15.907733 12662 catalog_manager.cc:4841] TS e583f83ff95447a0bd7a82de3b9a8e85 (127.1.245.67:38729): tablet f388d82bbe8a44b6a82dba34b498b3de (table test-table [id=742f53f08ca04c4a90217375bee021d3]) successfully deleted
I20260806 18:45:15.909381 13468 ts_tablet_manager.cc:1918] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 18:45:15.909456 13468 log.cc:1196] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/f388d82bbe8a44b6a82dba34b498b3de
I20260806 18:45:15.909701 13468 ts_tablet_manager.cc:1939] T f388d82bbe8a44b6a82dba34b498b3de P edff284cbe514cdb99e162e92d303825: Deleting consensus metadata
I20260806 18:45:15.910422 12658 catalog_manager.cc:4841] TS edff284cbe514cdb99e162e92d303825 (127.1.245.66:42777): tablet f388d82bbe8a44b6a82dba34b498b3de (table test-table [id=742f53f08ca04c4a90217375bee021d3]) successfully deleted
W20260806 18:45:15.914856 12658 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:38933 (error 108)
I20260806 18:45:15.915100 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 12988
I20260806 18:45:15.917781 13535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:15.915937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:38933 (local address 127.0.0.1:57232)
0806 18:45:15.917603 (+ 1666us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:15.917680 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:38933: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1579,"thread_start_us":1502,"threads_started":1}
W20260806 18:45:15.918859 12658 catalog_manager.cc:4584] TS 9a0dfc72c6fe460eb36872748f728337 (127.1.245.65:38933): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f388d82bbe8a44b6a82dba34b498b3de: Network error: Client connection negotiation failed: client connection to 127.1.245.65:38933: connect: Connection refused (error 111)
I20260806 18:45:15.928679 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 13175
I20260806 18:45:15.948800 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 12636
2026-08-06T18:45:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3834 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:45:16 dist-test-slave-fgb3 krb5kdc[13538](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:45:16 dist-test-slave-fgb3 krb5kdc[13538](info): set up 2 sockets
Aug 06 18:45:16 dist-test-slave-fgb3 krb5kdc[13538](info): commencing operation
krb5kdc: starting...
W20260806 18:45:17.070461 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:45:17 dist-test-slave-fgb3 krb5kdc[13538](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:45:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:45:17.197489 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:37621
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:37621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:17.308100 13553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:17.308399 13553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:17.308457 13553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:17.312515 13553 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:17.312583 13553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:17.312610 13553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:17.312631 13553 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:17.312657 13553 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:17.318976 13553 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:37621
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:37621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:17.320310 13553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:17.321382 13553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:17.329797 13561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:17.329849 13562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:17.329917 13553 server_base.cc:1034] running on GCE node
W20260806 18:45:17.329797 13564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:17.330703 13553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:17.331750 13553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:17.332909 13553 hybrid_clock.cc:648] HybridClock initialized: now 1786041917332894 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T18:45:17Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1392 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:45:17 dist-test-slave-fgb3 krb5kdc[13572](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:45:17 dist-test-slave-fgb3 krb5kdc[13572](info): set up 2 sockets
Aug 06 18:45:17 dist-test-slave-fgb3 krb5kdc[13572](info): commencing operation
krb5kdc: starting...
W20260806 18:45:18.465875 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:45:18 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:45:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:45:18.592877 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:34839
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:34839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:18.707403 13593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:18.707686 13593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:18.707746 13593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:18.711812 13593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:18.711922 13593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:18.711951 13593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:18.711974 13593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:18.711999 13593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:18.717378 13593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:34839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:34839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:18.718568 13593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:18.719596 13593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:18.728080 13604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:18.728137 13593 server_base.cc:1034] running on GCE node
W20260806 18:45:18.728173 13599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:18.728281 13600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:18.728895 13593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:18.729974 13593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:18.731148 13593 hybrid_clock.cc:648] HybridClock initialized: now 1786041918731128 us; error 29 us; skew 500 ppm
Aug 06 18:45:18 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041918, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:18.740895 13593 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:45:18.741962 13593 webserver.cc:460] Webserver started at http://127.1.245.126:34147/ using document root <none> and password file <none>
I20260806 18:45:18.742614 13593 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:18.742690 13593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:18.742911 13593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:18.745019 13593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "b8034ac8ecfd48b6a322168523dd86e1"
format_stamp: "Formatted at 2026-08-06 18:45:18 on dist-test-slave-fgb3"
server_key: "13e51f1340ec2ac8f1fcd2471a2fe3bb"
server_key_iv: "181691ce56e03a7ab0592970f83d858e"
server_key_version: "encryptionkey@0"
I20260806 18:45:18.745585 13593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "b8034ac8ecfd48b6a322168523dd86e1"
format_stamp: "Formatted at 2026-08-06 18:45:18 on dist-test-slave-fgb3"
server_key: "13e51f1340ec2ac8f1fcd2471a2fe3bb"
server_key_iv: "181691ce56e03a7ab0592970f83d858e"
server_key_version: "encryptionkey@0"
I20260806 18:45:18.749459 13593 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:45:18.752050 13613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:18.753372 13593 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:18.753495 13593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "b8034ac8ecfd48b6a322168523dd86e1"
format_stamp: "Formatted at 2026-08-06 18:45:18 on dist-test-slave-fgb3"
server_key: "13e51f1340ec2ac8f1fcd2471a2fe3bb"
server_key_iv: "181691ce56e03a7ab0592970f83d858e"
server_key_version: "encryptionkey@0"
I20260806 18:45:18.753603 13593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:18.767019 13593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:18.769834 13593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:18.770001 13593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:18.777132 13593 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:34839
I20260806 18:45:18.777161 13678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:34839 every 8 connection(s)
I20260806 18:45:18.778085 13593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:18.779879 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 13593
I20260806 18:45:18.779999 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:18.780294 2005 external_mini_cluster.cc:1428] Setting key 39cf35396ac600e2dbd6f86d3005c991
I20260806 18:45:18.781253 13679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:18.786955 13679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Bootstrap starting.
Aug 06 18:45:18 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:18.789392 13679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:18.790297 13679 log.cc:824] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:18.792668 13679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: No bootstrap required, opened a new log
I20260806 18:45:18.794870 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:18.781752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41460 (local address 127.1.245.126:34839)
0806 18:45:18.782215 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:18.782259 (+ 44us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:18.782294 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:45:18.782960 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:18.782977 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:18.783290 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:18.783579 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:18.784375 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:18.785346 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:18.786166 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:18.786673 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:18.786821 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:18.789471 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:18.789477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:18.789493 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:18.789529 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:18.791440 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:18.791914 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:18.791919 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:18.791925 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:18.792070 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:18.792311 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:18.792314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:18.792316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:18.792832 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:18.793033 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:18.793349 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 18:45:18.793651 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":159,"threads_started":1}
I20260806 18:45:18.796594 13679 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } }
I20260806 18:45:18.796852 13679 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:18.796909 13679 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8034ac8ecfd48b6a322168523dd86e1, State: Initialized, Role: FOLLOWER
I20260806 18:45:18.797353 13679 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } }
I20260806 18:45:18.797483 13679 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:18.797545 13679 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:18.797639 13679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:18.798614 13679 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } }
I20260806 18:45:18.798935 13679 leader_election.cc:302] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8034ac8ecfd48b6a322168523dd86e1; no voters:
I20260806 18:45:18.799222 13679 leader_election.cc:288] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:18.799367 13684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:18.799566 13684 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 1 LEADER]: Becoming Leader. State: Replica: b8034ac8ecfd48b6a322168523dd86e1, State: Running, Role: LEADER
I20260806 18:45:18.799851 13684 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } }
I20260806 18:45:18.800403 13679 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:18.800937 13684 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8034ac8ecfd48b6a322168523dd86e1. Latest consensus state: current_term: 1 leader_uuid: "b8034ac8ecfd48b6a322168523dd86e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } } }
I20260806 18:45:18.801059 13684 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:18.801658 13694 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:18.802454 13690 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8034ac8ecfd48b6a322168523dd86e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } } }
I20260806 18:45:18.802556 13690 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:18.805335 13694 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:18.811676 13694 catalog_manager.cc:1347] Generated new cluster ID: f2f87a0cd9f94e13a55b613ff8cb8b04
I20260806 18:45:18.811762 13694 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:18.822604 13694 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:18.823848 13694 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:18.837812 13694 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Generated new TSK 0
I20260806 18:45:18.838706 13694 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:45:18.875636 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34839
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:18.986181 13714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:18.986503 13714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:18.986555 13714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:18.990553 13714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:18.990612 13714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:18.990672 13714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:18.995874 13714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:18.996955 13714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:18.998040 13714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:19.007891 13721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:19.007926 13714 server_base.cc:1034] running on GCE node
W20260806 18:45:19.007895 13722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.008179 13724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:19.008622 13714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:19.009256 13714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:19.010402 13714 hybrid_clock.cc:648] HybridClock initialized: now 1786041919010371 us; error 42 us; skew 500 ppm
Aug 06 18:45:19 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041919, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:19.013113 13714 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:45:19.013972 13714 webserver.cc:460] Webserver started at http://127.1.245.65:42929/ using document root <none> and password file <none>
I20260806 18:45:19.014552 13714 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:19.014614 13714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:19.014791 13714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:19.016788 13714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "f632de78f84d4ead82b18fb04bf02121"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "6c419e705d99dc81b7a328b250b5752c"
server_key_iv: "f1850a9a913d341b8cb2f8cd66172b8e"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.017310 13714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "f632de78f84d4ead82b18fb04bf02121"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "6c419e705d99dc81b7a328b250b5752c"
server_key_iv: "f1850a9a913d341b8cb2f8cd66172b8e"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.021184 13714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:45:19.024029 13733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.025339 13714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:19.025449 13714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "f632de78f84d4ead82b18fb04bf02121"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "6c419e705d99dc81b7a328b250b5752c"
server_key_iv: "f1850a9a913d341b8cb2f8cd66172b8e"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.025543 13714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:19.045702 13714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:19.048900 13714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:19.049074 13714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:19.049661 13714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:19.050551 13714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:19.050602 13714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.050644 13714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:19.050664 13714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.059787 13714 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:39657
I20260806 18:45:19.059808 13885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:39657 every 8 connection(s)
I20260806 18:45:19.060783 13714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:19.062606 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 13714
I20260806 18:45:19.062731 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:45:19.063032 2005 external_mini_cluster.cc:1428] Setting key 466bb45a77b3f6ab9d8902987a9f5f06
Aug 06 18:45:19 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041919, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:45:19.076997 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:19.062875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:58229 (local address 127.1.245.126:34839)
0806 18:45:19.063041 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:19.063046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:19.064195 (+ 1149us) server_negotiation.cc:407] Connection header received
0806 18:45:19.065381 (+ 1186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:19.065383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:19.065433 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:19.065772 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:19.067414 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.068256 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.069527 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.069908 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.070025 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:19.073245 (+ 3220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:19.073249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:19.073251 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:19.073282 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:19.074793 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:19.075366 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:19.075368 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:19.075370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:19.075446 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:19.075726 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:19.075729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:19.075730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:19.075932 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:19.076008 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:19.076646 (+ 638us) server_negotiation.cc:299] Negotiation successful
0806 18:45:19.076818 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:45:19.077854 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:19.063311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:58229)
0806 18:45:19.064055 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:19.064099 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:19.065090 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:19.066000 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:19.066011 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:19.066571 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:19.067237 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:19.067253 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.068420 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.068423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:19.069380 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:19.069391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.070105 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.070108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:19.070273 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:19.070987 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:19.071012 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:19.073043 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:19.074908 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:19.074914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:19.074928 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:19.075276 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:19.075512 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:19.075514 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:19.075515 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:19.075651 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:19.076021 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:19.076027 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:19.076378 (+ 351us) client_negotiation.cc:769] Sending connection context
0806 18:45:19.076558 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 18:45:19.076918 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":139,"threads_started":1}
I20260806 18:45:19.079294 13886 heartbeater.cc:347] Connected to a master server at 127.1.245.126:34839
I20260806 18:45:19.079555 13886 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:19.080085 13886 heartbeater.cc:511] Master 127.1.245.126:34839 requested a full tablet report, sending...
I20260806 18:45:19.081638 13636 ts_manager.cc:194] Registered new tserver with Master: f632de78f84d4ead82b18fb04bf02121 (127.1.245.65:39657)
I20260806 18:45:19.083022 13636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:58229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:45:19.093451 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34839
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:19.206760 13894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:19.207046 13894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:19.207089 13894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:19.211143 13894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:19.211220 13894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:19.211308 13894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:19.216741 13894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13894
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:19.217878 13894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:19.218976 13894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:19.228632 13900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.228765 13901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.228628 13903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:19.229108 13894 server_base.cc:1034] running on GCE node
I20260806 18:45:19.229630 13894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:19.230376 13894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:19.231565 13894 hybrid_clock.cc:648] HybridClock initialized: now 1786041919231543 us; error 33 us; skew 500 ppm
Aug 06 18:45:19 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041919, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:19.234239 13894 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:45:19.235140 13894 webserver.cc:460] Webserver started at http://127.1.245.66:40285/ using document root <none> and password file <none>
I20260806 18:45:19.235682 13894 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:19.235749 13894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:19.235947 13894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:19.237934 13894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "d995b71f007a446d9b05c135784a2e77"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "9769e404c049951ddfc1498405c0a80c"
server_key_iv: "cdf4a3c431d251599e6f22a8f657a162"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.238545 13894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "d995b71f007a446d9b05c135784a2e77"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "9769e404c049951ddfc1498405c0a80c"
server_key_iv: "cdf4a3c431d251599e6f22a8f657a162"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.242234 13894 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:45:19.244729 13912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.245976 13894 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:19.246106 13894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "d995b71f007a446d9b05c135784a2e77"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "9769e404c049951ddfc1498405c0a80c"
server_key_iv: "cdf4a3c431d251599e6f22a8f657a162"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.246203 13894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:19.257704 13894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:19.261070 13894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:19.261262 13894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:19.261888 13894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:19.262781 13894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:19.262832 13894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.262887 13894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:19.262913 13894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.271512 13894 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:35955
I20260806 18:45:19.271538 14065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:35955 every 8 connection(s)
I20260806 18:45:19.272451 13894 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:19.280972 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 13894
I20260806 18:45:19.281129 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:45:19.281442 2005 external_mini_cluster.cc:1428] Setting key bd43ce2eea63bf37f5eb63ae2fea8226
Aug 06 18:45:19 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041919, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:19.286480 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:19.274405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:33837 (local address 127.1.245.126:34839)
0806 18:45:19.274535 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:19.274539 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:19.275672 (+ 1133us) server_negotiation.cc:407] Connection header received
0806 18:45:19.276548 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:19.276551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:19.276607 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:19.276692 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:19.277875 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.278530 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.279735 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.279959 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.280050 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:19.282803 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:19.282808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:19.282811 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:19.282839 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:19.284264 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:19.284743 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:19.284746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:19.284749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:19.284841 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:19.285203 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:19.285206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:19.285207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:19.285410 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:19.285488 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:19.286147 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 18:45:19.286271 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:45:19.287304 14069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:19.274784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:33837)
0806 18:45:19.275510 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:19.275560 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:19.276326 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:19.276845 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:19.276854 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:19.277207 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:19.277718 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:19.277731 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.278673 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.278676 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:19.279603 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:19.279613 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.280100 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.280102 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:19.280242 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:19.280807 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:19.280827 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:19.282636 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:19.284395 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:19.284400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:19.284411 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:19.284656 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:19.284933 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:19.284936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:19.284938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:19.285114 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:19.285506 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:19.285513 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:19.285845 (+ 332us) client_negotiation.cc:769] Sending connection context
0806 18:45:19.286063 (+ 218us) client_negotiation.cc:240] Negotiation successful
0806 18:45:19.286408 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":119,"threads_started":1}
I20260806 18:45:19.288453 14067 heartbeater.cc:347] Connected to a master server at 127.1.245.126:34839
I20260806 18:45:19.288687 14067 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:19.289173 14067 heartbeater.cc:511] Master 127.1.245.126:34839 requested a full tablet report, sending...
I20260806 18:45:19.290331 13636 ts_manager.cc:194] Registered new tserver with Master: d995b71f007a446d9b05c135784a2e77 (127.1.245.66:35955)
I20260806 18:45:19.290949 13636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:33837
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:45:19.309572 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34839
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:19.421393 14072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:19.421653 14072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:19.421711 14072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:19.425817 14072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:19.425877 14072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:19.425954 14072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:19.431187 14072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:34839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:19.432255 14072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:19.433281 14072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:45:19.443015 14072 server_base.cc:1034] running on GCE node
W20260806 18:45:19.443066 14083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.442976 14079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.442976 14080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:19.443588 14072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:19.444247 14072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:19.445437 14072 hybrid_clock.cc:648] HybridClock initialized: now 1786041919445404 us; error 40 us; skew 500 ppm
Aug 06 18:45:19 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041919, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:19.448215 14072 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:45:19.449147 14072 webserver.cc:460] Webserver started at http://127.1.245.67:46623/ using document root <none> and password file <none>
I20260806 18:45:19.449728 14072 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:19.449793 14072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:19.450039 14072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:19.452378 14072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "3d5d4cc19208476dbb1e315fb3419999"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "e6e64d19f821899e7a1f662a9b71b6b0"
server_key_iv: "bc747f92b883990044419398af5b4fc4"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.453018 14072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "3d5d4cc19208476dbb1e315fb3419999"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "e6e64d19f821899e7a1f662a9b71b6b0"
server_key_iv: "bc747f92b883990044419398af5b4fc4"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.457334 14072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:45:19.460081 14094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.461360 14072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:45:19.461477 14072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "3d5d4cc19208476dbb1e315fb3419999"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "e6e64d19f821899e7a1f662a9b71b6b0"
server_key_iv: "bc747f92b883990044419398af5b4fc4"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.461576 14072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:19.480808 14072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:19.484031 14072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:19.484211 14072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:19.484812 14072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:19.485677 14072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:19.485728 14072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.485780 14072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:19.485807 14072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.494444 14072 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:42473
I20260806 18:45:19.494467 14230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:42473 every 8 connection(s)
I20260806 18:45:19.495327 14072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:19.496372 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 14072
I20260806 18:45:19.496506 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:19.496755 2005 external_mini_cluster.cc:1428] Setting key cccc6733d20ba3b450354c00b15b9c9a
Aug 06 18:45:19 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041919, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:19.508366 13682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:19.497194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:58177 (local address 127.1.245.126:34839)
0806 18:45:19.497342 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:19.497346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:19.498291 (+ 945us) server_negotiation.cc:407] Connection header received
0806 18:45:19.499626 (+ 1335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:19.499628 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:19.499673 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:19.499735 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:19.500864 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.501517 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.502709 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.502918 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.502998 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:19.505242 (+ 2244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:19.505245 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:19.505247 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:19.505272 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:19.506573 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:19.507018 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:19.507020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:19.507022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:19.507095 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:19.507399 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:19.507401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:19.507403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:19.507585 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:19.507660 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:19.508144 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 18:45:19.508242 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:45:19.509225 14234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:19.497598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:58177)
0806 18:45:19.498138 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:19.498191 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:19.499442 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:19.499888 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:19.499894 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:19.500256 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:19.500744 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:19.500757 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.501620 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.501623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:19.502619 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:19.502629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:19.503038 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:19.503041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:19.503178 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:19.503725 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:19.503741 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:19.505101 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:19.506681 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:19.506686 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:19.506698 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:19.506943 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:19.507154 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:19.507156 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:19.507157 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:19.507285 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:19.507682 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:19.507688 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:19.507949 (+ 261us) client_negotiation.cc:769] Sending connection context
0806 18:45:19.508106 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 18:45:19.508377 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":111,"threads_started":1}
I20260806 18:45:19.510227 14231 heartbeater.cc:347] Connected to a master server at 127.1.245.126:34839
I20260806 18:45:19.510509 14231 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:19.511122 14231 heartbeater.cc:511] Master 127.1.245.126:34839 requested a full tablet report, sending...
I20260806 18:45:19.512184 13636 ts_manager.cc:194] Registered new tserver with Master: 3d5d4cc19208476dbb1e315fb3419999 (127.1.245.67:42473)
I20260806 18:45:19.512741 13636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:58177
I20260806 18:45:19.522477 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab.
Entry for principal oryx/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab.
WARNING: no policy specified for oryx/127.1.245.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab.
Entry for principal oryx/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab.
WARNING: no policy specified for oryx/127.1.245.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab.
Entry for principal oryx/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab.
I20260806 18:45:19.600482 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 13714
I20260806 18:45:19.623827 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 13894
I20260806 18:45:19.632359 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 14072
I20260806 18:45:19.641673 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 13593
I20260806 18:45:19.654022 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:34839
--webserver_interface=127.1.245.126
--webserver_port=34147
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:34839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:19.765856 14242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:19.766122 14242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:19.766177 14242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:19.770169 14242 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:19.770228 14242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:19.770253 14242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:19.770274 14242 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:19.770294 14242 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:19.775565 14242 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:34839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:34839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=34147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:19.776664 14242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:19.777709 14242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:45:19.786412 14242 server_base.cc:1034] running on GCE node
W20260806 18:45:19.786371 14253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.786474 14249 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.786371 14255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:19.787245 14242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:19.788395 14242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:19.789546 14242 hybrid_clock.cc:648] HybridClock initialized: now 1786041919789525 us; error 32 us; skew 500 ppm
Aug 06 18:45:19 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041919, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:19.792387 14242 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:45:19.793298 14242 webserver.cc:460] Webserver started at http://127.1.245.126:34147/ using document root <none> and password file <none>
I20260806 18:45:19.793849 14242 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:19.793932 14242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:19.797909 14242 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:45:19.800283 14264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:19.801625 14242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:45:19.801749 14242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "b8034ac8ecfd48b6a322168523dd86e1"
format_stamp: "Formatted at 2026-08-06 18:45:18 on dist-test-slave-fgb3"
server_key: "13e51f1340ec2ac8f1fcd2471a2fe3bb"
server_key_iv: "181691ce56e03a7ab0592970f83d858e"
server_key_version: "encryptionkey@0"
I20260806 18:45:19.802289 14242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:19.821530 14242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:19.824659 14242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:19.824837 14242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:19.831914 14242 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:34839
I20260806 18:45:19.831959 14338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:34839 every 8 connection(s)
I20260806 18:45:19.833035 14242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:19.837461 14339 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:45:19.839174 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Bootstrap starting.
I20260806 18:45:19.840453 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 14242
I20260806 18:45:19.840926 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:39657
--local_ip_for_outbound_sockets=127.1.245.65
--tserver_master_addrs=127.1.245.126:34839
--webserver_port=42929
--webserver_interface=127.1.245.65
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.245.65
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:45:19.856139 14339 log.cc:824] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:19.862960 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:45:19.863297 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Bootstrap complete.
I20260806 18:45:19.867488 14339 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } }
I20260806 18:45:19.867892 14339 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8034ac8ecfd48b6a322168523dd86e1, State: Initialized, Role: FOLLOWER
I20260806 18:45:19.868353 14339 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } }
I20260806 18:45:19.868480 14339 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:19.868546 14339 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:19.868639 14339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:45:19.870584 14339 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } }
I20260806 18:45:19.870924 14339 leader_election.cc:302] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8034ac8ecfd48b6a322168523dd86e1; no voters:
I20260806 18:45:19.871224 14339 leader_election.cc:288] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:45:19.871389 14343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:45:19.871609 14343 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [term 2 LEADER]: Becoming Leader. State: Replica: b8034ac8ecfd48b6a322168523dd86e1, State: Running, Role: LEADER
I20260806 18:45:19.871982 14343 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } }
I20260806 18:45:19.872555 14339 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:19.873430 14346 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8034ac8ecfd48b6a322168523dd86e1. Latest consensus state: current_term: 2 leader_uuid: "b8034ac8ecfd48b6a322168523dd86e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } } }
I20260806 18:45:19.873553 14346 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:19.873860 14344 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b8034ac8ecfd48b6a322168523dd86e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8034ac8ecfd48b6a322168523dd86e1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 34839 } } }
I20260806 18:45:19.873939 14344 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:19.873963 14355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:19.877274 14355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:19.877949 14355 catalog_manager.cc:1259] Loaded cluster ID: f2f87a0cd9f94e13a55b613ff8cb8b04
I20260806 18:45:19.878016 14355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:19.879750 14355 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:19.880308 14355 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8034ac8ecfd48b6a322168523dd86e1: Loaded TSK: 0
I20260806 18:45:19.881213 14355 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:45:19.966288 14341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:19.966718 14341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:19.966804 14341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:19.972710 14341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:19.972812 14341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:19.972916 14341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:19.979578 14341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.65.keytab
--principal=oryx/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:39657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=42929
--tserver_master_addrs=127.1.245.126:34839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:19.980710 14341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:19.981743 14341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:19.992277 14373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.992295 14372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:19.992295 14377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:19.992473 14341 server_base.cc:1034] running on GCE node
I20260806 18:45:19.993184 14341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:19.993957 14341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:19.995122 14341 hybrid_clock.cc:648] HybridClock initialized: now 1786041919995098 us; error 39 us; skew 500 ppm
Aug 06 18:45:19 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041919, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:19.997889 14341 init.cc:377] Logged in from keytab as oryx/127.1.245.65@KRBTEST.COM (short username oryx)
I20260806 18:45:19.999022 14341 webserver.cc:460] Webserver started at http://127.1.245.65:42929/ using document root <none> and password file <none>
I20260806 18:45:19.999610 14341 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:19.999697 14341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:20.004040 14341 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:45:20.006415 14385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.007786 14341 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:45:20.007910 14341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "f632de78f84d4ead82b18fb04bf02121"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "6c419e705d99dc81b7a328b250b5752c"
server_key_iv: "f1850a9a913d341b8cb2f8cd66172b8e"
server_key_version: "encryptionkey@0"
I20260806 18:45:20.008505 14341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:20.022467 14341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:20.025810 14341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:20.026003 14341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:20.026680 14341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:20.027544 14341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:20.027593 14341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.027647 14341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:20.027673 14341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.036011 14341 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:39657
I20260806 18:45:20.036036 14554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:39657 every 8 connection(s)
I20260806 18:45:20.036993 14341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:20.038120 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 14341
I20260806 18:45:20.038687 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:35955
--local_ip_for_outbound_sockets=127.1.245.66
--tserver_master_addrs=127.1.245.126:34839
--webserver_port=40285
--webserver_interface=127.1.245.66
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.245.66
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:45:20.051074 14560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.039421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:49683 (local address 127.1.245.126:34839)
0806 18:45:20.040038 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.040059 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.040352 (+ 293us) server_negotiation.cc:407] Connection header received
0806 18:45:20.041412 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.041432 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.041815 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.042151 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.043359 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.044891 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.046083 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.046858 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.047034 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.050447 (+ 3413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:49683: BlockingRecv error: recv got EOF from 127.1.245.65:49683 (error 108)
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":122,"threads_started":1}
I20260806 18:45:20.051117 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.039467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:49683)
0806 18:45:20.040162 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.040213 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.041029 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.042278 (+ 1249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.042285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.042698 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.043168 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.043184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.045043 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.045046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.045947 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.045957 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.047039 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.047043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.047179 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.047728 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.047744 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.050188 (+ 2444us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":107,"threads_started":1}
W20260806 18:45:20.051546 14555 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:34839 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.245.126:34839: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:20.056869 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.051960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:42293)
0806 18:45:20.052069 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.052083 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.052155 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.052348 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.052351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.052514 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.052617 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.052623 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.053279 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.053282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.054152 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.054161 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.054587 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.054590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.054680 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.055058 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.055070 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.056744 (+ 1674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:45:20.057240 14560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.052071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:42293 (local address 127.1.245.126:34839)
0806 18:45:20.052173 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.052177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.052189 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:45:20.052219 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.052221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.052256 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.052321 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.052703 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.053179 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.054246 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.054496 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.054573 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.056875 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:42293: BlockingRecv error: recv got EOF from 127.1.245.65:42293 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:20.062727 14558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.057371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:34643)
0806 18:45:20.057524 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.057537 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.057599 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.057783 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.057786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.057931 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.058023 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.058028 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.058788 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.058791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.059657 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.059668 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.060123 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.060127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.060206 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.060621 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.060634 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.062598 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97}
W20260806 18:45:20.062844 14560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.057512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:34643 (local address 127.1.245.126:34839)
0806 18:45:20.057604 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.057607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.057618 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:45:20.057665 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.057666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.057700 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.057758 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.058154 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.058696 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.059793 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.060018 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.060107 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.062742 (+ 2635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:34643: BlockingRecv error: recv got EOF from 127.1.245.65:34643 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260806 18:45:20.063063 14555 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:45:20.155786 14561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:20.156055 14561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:20.156098 14561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:20.160444 14561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:20.160514 14561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:20.160576 14561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:20.166477 14561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.66.keytab
--principal=oryx/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:35955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=40285
--tserver_master_addrs=127.1.245.126:34839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:20.167591 14561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:20.168672 14561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:20.178121 14571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:20.178117 14567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:20.178242 14568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:20.178445 14561 server_base.cc:1034] running on GCE node
I20260806 18:45:20.178876 14561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:20.179489 14561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:20.180649 14561 hybrid_clock.cc:648] HybridClock initialized: now 1786041920180620 us; error 39 us; skew 500 ppm
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041920, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:20.183382 14561 init.cc:377] Logged in from keytab as oryx/127.1.245.66@KRBTEST.COM (short username oryx)
I20260806 18:45:20.184206 14561 webserver.cc:460] Webserver started at http://127.1.245.66:40285/ using document root <none> and password file <none>
I20260806 18:45:20.184723 14561 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:20.184798 14561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:20.188519 14561 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.002s
I20260806 18:45:20.190567 14579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.191776 14561 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:20.191882 14561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "d995b71f007a446d9b05c135784a2e77"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "9769e404c049951ddfc1498405c0a80c"
server_key_iv: "cdf4a3c431d251599e6f22a8f657a162"
server_key_version: "encryptionkey@0"
I20260806 18:45:20.192421 14561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:20.217213 14561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:20.220580 14561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:20.220763 14561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:20.221405 14561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:20.222271 14561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:20.222339 14561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.222404 14561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:20.222433 14561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.231110 14561 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:35955
I20260806 18:45:20.231146 14727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:35955 every 8 connection(s)
I20260806 18:45:20.232092 14561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:20.236516 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 14561
I20260806 18:45:20.237020 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:42473
--local_ip_for_outbound_sockets=127.1.245.67
--tserver_master_addrs=127.1.245.126:34839
--webserver_port=46623
--webserver_interface=127.1.245.67
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.245.67
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:45:20.243484 14560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.233841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:46871 (local address 127.1.245.126:34839)
0806 18:45:20.233985 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.233990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.234936 (+ 946us) server_negotiation.cc:407] Connection header received
0806 18:45:20.236156 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.236159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.236213 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.236291 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.237823 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.238411 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.239568 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.239874 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.239960 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.243333 (+ 3373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:46871: BlockingRecv error: recv got EOF from 127.1.245.66:46871 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:45:20.244059 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.234233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:46871)
0806 18:45:20.234781 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.234824 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.235962 (+ 1138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.236506 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.236515 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.236987 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.237656 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.237676 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.238545 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.238549 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.239446 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.239455 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.240012 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.240015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.240165 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.240749 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.240766 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.243140 (+ 2374us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":118,"threads_started":1}
W20260806 18:45:20.244587 14728 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:34839 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.245.126:34839: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:20.249943 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.245118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:56611)
0806 18:45:20.245232 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.245246 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.245328 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.245576 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.245579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.245724 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.245812 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.245817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.246590 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.246593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.247120 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.247128 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.247549 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.247553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.247639 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.248063 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.248074 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.249796 (+ 1722us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 18:45:20.250038 14560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.245205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:56611 (local address 127.1.245.126:34839)
0806 18:45:20.245314 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.245317 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.245330 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:20.245439 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.245442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.245487 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.245550 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.245937 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.246485 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.247224 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.247453 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.247540 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.249932 (+ 2392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:56611: BlockingRecv error: recv got EOF from 127.1.245.66:56611 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:20.255092 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.250594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:34701)
0806 18:45:20.250694 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.250708 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.250782 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.251003 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.251006 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.251135 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.251201 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.251205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.251940 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.251943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.252436 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.252443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.252852 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.252856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.252929 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.253328 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.253339 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.254976 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 18:45:20.255304 14560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.250668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:34701 (local address 127.1.245.126:34839)
0806 18:45:20.250769 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.250773 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.250785 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:45:20.250874 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.250877 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.250919 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.250979 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.251301 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.251838 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.252538 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.252751 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.252832 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.255100 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:34701: BlockingRecv error: recv got EOF from 127.1.245.66:34701 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 18:45:20.255370 14728 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:45:20.351786 14732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:20.352046 14732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:20.352105 14732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:20.356038 14732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:20.356099 14732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:20.356178 14732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:20.361380 14732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/krb5kdc/oryx_127.1.245.67.keytab
--principal=oryx/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:42473
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=46623
--tserver_master_addrs=127.1.245.126:34839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:20.362501 14732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:20.363538 14732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:20.373034 14742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:20.373039 14740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:20.373034 14738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:20.373497 14732 server_base.cc:1034] running on GCE node
I20260806 18:45:20.374058 14732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:20.374893 14732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:20.376055 14732 hybrid_clock.cc:648] HybridClock initialized: now 1786041920376030 us; error 37 us; skew 500 ppm
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041920, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:20.378899 14732 init.cc:377] Logged in from keytab as oryx/127.1.245.67@KRBTEST.COM (short username oryx)
I20260806 18:45:20.379824 14732 webserver.cc:460] Webserver started at http://127.1.245.67:46623/ using document root <none> and password file <none>
I20260806 18:45:20.380368 14732 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:20.380443 14732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:20.384274 14732 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260806 18:45:20.386406 14750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.387607 14732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 18:45:20.387706 14732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "3d5d4cc19208476dbb1e315fb3419999"
format_stamp: "Formatted at 2026-08-06 18:45:19 on dist-test-slave-fgb3"
server_key: "e6e64d19f821899e7a1f662a9b71b6b0"
server_key_iv: "bc747f92b883990044419398af5b4fc4"
server_key_version: "encryptionkey@0"
I20260806 18:45:20.388316 14732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:20.407594 14732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:20.410517 14732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:20.410696 14732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:20.411307 14732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:20.412213 14732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:20.412261 14732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.412303 14732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:20.412331 14732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:20.421344 14732 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:42473
I20260806 18:45:20.421382 14902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:42473 every 8 connection(s)
I20260806 18:45:20.422348 14732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:20.424372 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 14732
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 18:45:20.433779 14560 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.424157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:46003 (local address 127.1.245.126:34839)
0806 18:45:20.424342 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.424346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.425504 (+ 1158us) server_negotiation.cc:407] Connection header received
0806 18:45:20.426652 (+ 1148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.426655 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.426690 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.426749 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.427997 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.428587 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.430107 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.430385 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.430486 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.433647 (+ 3161us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:46003: BlockingRecv error: recv got EOF from 127.1.245.67:46003 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260806 18:45:20.434293 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.425376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.1.245.126:34839)
0806 18:45:20.425627 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.425630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.425638 (+ 8us) server_negotiation.cc:407] Connection header received
0806 18:45:20.425673 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.425675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.425714 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.425791 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.426725 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.427274 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.428382 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.428708 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.428780 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.430294 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:20.430301 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:20.430344 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:20.430378 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:20.432242 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:20.432657 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:20.432664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:20.432670 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:20.432818 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:20.433104 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:20.433106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:20.433107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:20.433459 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:20.433649 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:20.433951 (+ 302us) server_negotiation.cc:299] Negotiation successful
0806 18:45:20.434178 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":98,"threads_started":1}
I20260806 18:45:20.434661 14906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.424574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:46003)
0806 18:45:20.425349 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.425406 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.426448 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.426905 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.426911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.427225 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.427853 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.427873 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.428705 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.428707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.429988 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.429999 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.430519 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.430522 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.430666 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.431085 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.431100 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.433519 (+ 2419us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":111,"threads_started":1}
W20260806 18:45:20.435103 14903 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:34839 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.245.126:34839: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:20.440541 14906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.435504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:49715)
0806 18:45:20.435945 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.435957 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.436038 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.436423 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.436426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.436563 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.436648 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.436652 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.437334 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.437338 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.437854 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.437861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.438287 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.438291 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.438402 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.438787 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.438798 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.440446 (+ 1648us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378}
W20260806 18:45:20.440670 14908 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.435596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:49715 (local address 127.1.245.126:34839)
0806 18:45:20.435690 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.435694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.436021 (+ 327us) server_negotiation.cc:407] Connection header received
0806 18:45:20.436281 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.436283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.436321 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.436386 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.436752 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.437232 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.437958 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.438181 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.438265 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.440566 (+ 2301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:49715: BlockingRecv error: recv got EOF from 127.1.245.67:49715 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:20 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:20.445562 14906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:20.441065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:53621)
0806 18:45:20.441171 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:20.441183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:20.441261 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:20.441491 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:20.441493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:20.441616 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:20.441690 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.441695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.442442 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.442445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.442957 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:20.442965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.443386 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.443389 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:20.443446 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.443843 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:20.443854 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:20.445473 (+ 1619us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:45:20.445712 14908 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:20.441142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:53621 (local address 127.1.245.126:34839)
0806 18:45:20.441317 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:20.441319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:20.441331 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:45:20.441358 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:20.441361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:20.441393 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:20.441451 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:20.441778 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.442218 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.443058 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:20.443254 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:20.443359 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:20.445605 (+ 2246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:53621: BlockingRecv error: recv got EOF from 127.1.245.67:53621 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260806 18:45:20.445791 14903 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 18:45:21 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:21 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:21.068908 14909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:21.063691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:35963)
0806 18:45:21.063991 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:21.064004 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:21.064091 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:21.064357 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:21.064359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:21.064531 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:21.064648 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:21.064653 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.065420 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.065423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:21.065998 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:21.066007 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.066507 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.066512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:21.066607 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:21.067053 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:21.067064 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:21.068795 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":74,"threads_started":1}
W20260806 18:45:21.069067 14910 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:21.063820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:35963 (local address 127.1.245.126:34839)
0806 18:45:21.064039 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:21.064042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:21.064057 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:45:21.064166 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:21.064168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:21.064221 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:21.064333 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:21.064750 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.065307 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.066100 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.066374 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.066452 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:21.068943 (+ 2491us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:35963: BlockingRecv error: recv got EOF from 127.1.245.65:35963 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":66,"threads_started":1}
Aug 06 18:45:21 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:21 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:21.261757 14911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:21.255998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:40491)
0806 18:45:21.256278 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:21.256292 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:21.256402 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:21.256721 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:21.256724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:21.256961 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:21.257093 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:21.257101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.258018 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.258022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:21.258716 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:21.258731 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.259215 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.259219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:21.259328 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:21.259774 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:21.259786 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:21.261602 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":72,"threads_started":1}
W20260806 18:45:21.261912 14910 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:21.256103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:40491 (local address 127.1.245.126:34839)
0806 18:45:21.256263 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:21.256266 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:21.256373 (+ 107us) server_negotiation.cc:407] Connection header received
0806 18:45:21.256537 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:21.256541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:21.256599 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:21.256682 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:21.257214 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.257864 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.258855 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.259074 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.259154 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:21.261776 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:40491: BlockingRecv error: recv got EOF from 127.1.245.66:40491 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 18:45:21 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:21 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:21.451825 14912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:21.446476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:48559)
0806 18:45:21.446740 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:21.446752 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:21.446857 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:21.447083 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:21.447085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:21.447262 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:21.447414 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:21.447421 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.448163 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.448166 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:21.448757 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:21.448767 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.449194 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.449196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:21.449257 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:21.449816 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:21.449828 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:21.451663 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":73,"threads_started":1}
W20260806 18:45:21.451910 14910 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:21.446569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:48559 (local address 127.1.245.126:34839)
0806 18:45:21.446696 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:21.446701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:21.446818 (+ 117us) server_negotiation.cc:407] Connection header received
0806 18:45:21.446931 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:21.446934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:21.446981 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:21.447050 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:21.447534 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.448063 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.448875 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:21.449095 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:21.449178 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:21.451806 (+ 2628us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:48559: BlockingRecv error: recv got EOF from 127.1.245.67:48559 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 18:45:22 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:22 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:22.075575 14914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:22.069793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:60769)
0806 18:45:22.070104 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:22.070119 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:22.070213 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:22.070539 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:22.070542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:22.070738 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:22.070869 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:22.070875 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.071668 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.071671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:22.072249 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:22.072258 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.072801 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.072804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:22.072875 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:22.073413 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:22.073430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:22.075453 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":95,"threads_started":1}
W20260806 18:45:22.075735 14915 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:22.069930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:60769 (local address 127.1.245.126:34839)
0806 18:45:22.070170 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:22.070173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:22.070187 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:22.070299 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:22.070301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:22.070375 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:22.070462 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:22.070997 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.071552 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.072382 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.072653 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.072742 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:22.075592 (+ 2850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:60769: BlockingRecv error: recv got EOF from 127.1.245.65:60769 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
Aug 06 18:45:22 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:22 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:22.268106 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:22.262646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:37969)
0806 18:45:22.262937 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:22.262950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:22.263062 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:22.263308 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:22.263310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:22.263493 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:22.263593 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:22.263598 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.264395 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.264397 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:22.264950 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:22.264958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.265426 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.265429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:22.265517 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:22.266023 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:22.266035 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:22.267962 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":92,"threads_started":1}
W20260806 18:45:22.268215 14915 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:22.262751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:37969 (local address 127.1.245.126:34839)
0806 18:45:22.262935 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:22.262938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:22.263009 (+ 71us) server_negotiation.cc:407] Connection header received
0806 18:45:22.263139 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:22.263141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:22.263196 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:22.263269 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:22.263745 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.264300 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.265067 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.265301 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.265410 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:22.268099 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:37969: BlockingRecv error: recv got EOF from 127.1.245.66:37969 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 06 18:45:22 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:22 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:22.458253 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:22.452767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:53255)
0806 18:45:22.453096 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:22.453111 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:22.453214 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:22.453560 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:22.453563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:22.453744 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:22.453857 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:22.453863 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.454724 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.454727 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:22.455293 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:22.455303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.455733 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.455737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:22.455830 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:22.456334 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:22.456346 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:22.458069 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":94,"threads_started":1}
W20260806 18:45:22.458343 14915 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:22.452997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:53255 (local address 127.1.245.126:34839)
0806 18:45:22.453160 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:22.453163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:22.453179 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:45:22.453359 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:22.453362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:22.453414 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:22.453485 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:22.453977 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.454601 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.455407 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:22.455624 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:22.455712 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:22.458214 (+ 2502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:53255: BlockingRecv error: recv got EOF from 127.1.245.67:53255 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 18:45:23 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:23 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:23.082157 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:23.076342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:46017)
0806 18:45:23.076655 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:23.076669 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:23.076778 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:23.077066 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:23.077068 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:23.077252 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:23.077389 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:23.077395 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.078295 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.078299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:23.078983 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:23.078996 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.079497 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.079501 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:23.079583 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:23.080050 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:23.080062 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:23.082019 (+ 1957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
W20260806 18:45:23.082430 14920 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:23.076432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:46017 (local address 127.1.245.126:34839)
0806 18:45:23.076673 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:23.076676 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:23.076743 (+ 67us) server_negotiation.cc:407] Connection header received
0806 18:45:23.076899 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:23.076902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:23.076959 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:23.077040 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:23.077509 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.078172 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.079098 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.079372 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.079478 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:23.082205 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:46017: BlockingRecv error: recv got EOF from 127.1.245.65:46017 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":66,"threads_started":1}
Aug 06 18:45:23 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:23 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:23.274456 14921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:23.268821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:55079)
0806 18:45:23.269140 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:23.269155 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:23.269275 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:23.269543 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:23.269546 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:23.269753 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:23.269857 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:23.269864 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.270688 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.270692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:23.271244 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:23.271253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.271707 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.271711 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:23.271796 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:23.272356 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:23.272369 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:23.274229 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":77,"threads_started":1}
W20260806 18:45:23.274514 14920 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:23.268904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:55079 (local address 127.1.245.126:34839)
0806 18:45:23.269071 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:23.269076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:23.269229 (+ 153us) server_negotiation.cc:407] Connection header received
0806 18:45:23.269377 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:23.269380 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:23.269430 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:23.269500 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:23.269980 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.270581 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.271367 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.271597 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.271684 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:23.274409 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:55079: BlockingRecv error: recv got EOF from 127.1.245.66:55079 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 18:45:23 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:23 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:23.464464 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:23.459071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:45519)
0806 18:45:23.459379 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:23.459392 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:23.459500 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:23.459784 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:23.459786 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:23.459963 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:23.460070 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:23.460076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.460851 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.460854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:23.461421 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:23.461430 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.461846 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.461849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:23.461907 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:23.462453 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:23.462466 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:23.464337 (+ 1871us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":91,"threads_started":1}
W20260806 18:45:23.464735 14920 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:23.459202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:45519 (local address 127.1.245.126:34839)
0806 18:45:23.459369 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:23.459373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:23.459460 (+ 87us) server_negotiation.cc:407] Connection header received
0806 18:45:23.459622 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:23.459626 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:23.459682 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:23.459759 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:23.460157 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.460730 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.461534 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:23.461755 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:23.461843 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:23.464605 (+ 2762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:45519: BlockingRecv error: recv got EOF from 127.1.245.67:45519 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 06 18:45:24 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:24 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:24.094521 14923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:24.083143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:48021)
0806 18:45:24.083476 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:24.083492 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:24.083584 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:24.083869 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:24.083872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:24.084096 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:24.084226 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:24.084234 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.084993 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.084996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:24.085568 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:24.085577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.086026 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.086029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:24.086094 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:24.092412 (+ 6318us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:24.092427 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:24.094360 (+ 1933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":123,"threads_started":1}
W20260806 18:45:24.094765 14924 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:24.083206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:48021 (local address 127.1.245.126:34839)
0806 18:45:24.083533 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:24.083537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:24.083553 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:45:24.083678 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:24.083681 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:24.083734 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:24.083826 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:24.084338 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.084870 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.085689 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.085923 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.086001 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:24.094525 (+ 8524us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:48021: BlockingRecv error: recv got EOF from 127.1.245.65:48021 (error 108)
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":57,"threads_started":1}
Aug 06 18:45:24 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:24 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:24.280705 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:24.275355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:48467)
0806 18:45:24.275637 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:24.275651 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:24.275753 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:24.275982 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:24.275984 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:24.276162 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:24.276261 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:24.276266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.277030 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.277033 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:24.277602 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:24.277611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.278087 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.278089 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:24.278150 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:24.278735 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:24.278752 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:24.280580 (+ 1828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":96,"threads_started":1}
W20260806 18:45:24.280907 14924 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:24.275450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:48467 (local address 127.1.245.126:34839)
0806 18:45:24.275596 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:24.275599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:24.275744 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:45:24.275830 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:24.275833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:24.275883 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:24.275960 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:24.276373 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.276924 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.277726 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.277988 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.278074 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:24.280768 (+ 2694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:48467: BlockingRecv error: recv got EOF from 127.1.245.66:48467 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 18:45:24 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:24 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:24.471840 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:24.465553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:38077)
0806 18:45:24.465876 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:24.465891 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:24.466014 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:24.466302 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:24.466306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:24.466523 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:24.466644 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:24.466650 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.467673 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.467677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:24.468536 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:24.468547 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.469124 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.469128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:24.469209 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:24.469779 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:24.469794 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:24.471685 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":102,"threads_started":1}
W20260806 18:45:24.471927 14924 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:24.465675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:38077 (local address 127.1.245.126:34839)
0806 18:45:24.465872 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:24.465877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:24.465966 (+ 89us) server_negotiation.cc:407] Connection header received
0806 18:45:24.466103 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:24.466106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:24.466156 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:24.466222 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:24.466751 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.467558 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.468672 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:24.469006 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:24.469106 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:24.471810 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:38077: BlockingRecv error: recv got EOF from 127.1.245.67:38077 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 18:45:25 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:25 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:25.101344 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:25.095454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:36417)
0806 18:45:25.095766 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:25.095783 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:25.095879 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:25.096166 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:25.096169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:25.096365 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:25.096465 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:25.096470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.097317 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.097321 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:25.097897 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:25.097907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.098525 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.098528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:25.098599 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:25.099162 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:25.099174 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:25.101215 (+ 2041us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":86,"threads_started":1}
W20260806 18:45:25.101465 14928 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:25.095546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:36417 (local address 127.1.245.126:34839)
0806 18:45:25.095837 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:25.095841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:25.095857 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:45:25.095962 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:25.095964 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:25.096012 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:25.096131 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:25.096596 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.097154 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.098035 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.098274 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.098382 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:25.101343 (+ 2961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:36417: BlockingRecv error: recv got EOF from 127.1.245.65:36417 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":91,"threads_started":1}
Aug 06 18:45:25 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:25 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:25.287019 14929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:25.281577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:39221)
0806 18:45:25.281911 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:25.281929 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:25.282036 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:25.282295 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:25.282298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:25.282534 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:25.282627 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:25.282633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.283388 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.283392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:25.283941 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:25.283948 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.284372 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.284376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:25.284458 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:25.284962 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:25.284972 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:25.286893 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":98,"threads_started":1}
W20260806 18:45:25.287267 14928 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:25.281776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:39221 (local address 127.1.245.126:34839)
0806 18:45:25.281963 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:25.281967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:25.281982 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:45:25.282132 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:25.282136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:25.282182 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:25.282248 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:25.282740 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.283280 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.284049 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.284272 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.284360 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:25.287046 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:39221: BlockingRecv error: recv got EOF from 127.1.245.66:39221 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 06 18:45:25 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:25 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:25.478735 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:25.473060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:57483)
0806 18:45:25.473364 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:25.473379 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:25.473477 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:25.473729 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:25.473732 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:25.473939 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:25.474044 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:25.474051 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.474861 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.474865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:25.475473 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:25.475485 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.476001 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.476005 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:25.476096 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:25.476652 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:25.476668 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:25.478606 (+ 1938us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":78,"threads_started":1}
W20260806 18:45:25.478845 14928 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:25.473331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:57483 (local address 127.1.245.126:34839)
0806 18:45:25.473457 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:25.473461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:25.473474 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:25.473563 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:25.473566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:25.473610 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:25.473675 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:25.474139 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.474727 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.475594 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:25.475853 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:25.475933 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:25.478744 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:57483: BlockingRecv error: recv got EOF from 127.1.245.67:57483 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 18:45:26 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:26 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:26.107700 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:26.102203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:40561)
0806 18:45:26.102534 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:26.102548 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:26.102636 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:26.102880 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:26.102882 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:26.103069 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:26.103167 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:26.103172 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.104007 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.104010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:26.104565 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:26.104575 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.105019 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.105021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:26.105086 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:26.105646 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:26.105658 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:26.107584 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":141,"threads_started":1}
W20260806 18:45:26.107861 14936 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:26.102348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:40561 (local address 127.1.245.126:34839)
0806 18:45:26.102577 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:26.102581 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:26.102595 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:26.102712 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:26.102715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:26.102775 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:26.102854 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:26.103299 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.103863 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.104677 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.104918 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.104999 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:26.107726 (+ 2727us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:40561: BlockingRecv error: recv got EOF from 127.1.245.65:40561 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":69,"threads_started":1}
Aug 06 18:45:26 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:26 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:26.293784 14937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:26.287905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:60923)
0806 18:45:26.288204 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:26.288217 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:26.288328 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:26.288581 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:26.288583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:26.288813 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:26.288936 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:26.288944 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.289918 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.289921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:26.290677 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:26.290688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.291096 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.291099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:26.291160 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:26.291655 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:26.291666 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:26.293656 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":108,"threads_started":1}
W20260806 18:45:26.294008 14936 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:26.288041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:60923 (local address 127.1.245.126:34839)
0806 18:45:26.288208 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:26.288212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:26.288335 (+ 123us) server_negotiation.cc:407] Connection header received
0806 18:45:26.288416 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:26.288419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:26.288471 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:26.288556 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:26.289090 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.289804 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.290778 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.291005 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.291086 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:26.293801 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:60923: BlockingRecv error: recv got EOF from 127.1.245.66:60923 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 06 18:45:26 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:26 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:26.484931 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:26.479543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:41289)
0806 18:45:26.479840 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:26.479854 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:26.479966 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:26.480211 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:26.480214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:26.480428 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:26.480524 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:26.480531 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.481267 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.481271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:26.481838 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:26.481847 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.482277 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.482281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:26.482399 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:26.482882 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:26.482895 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:26.484810 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":83,"threads_started":1}
W20260806 18:45:26.485092 14936 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:26.479638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:41289 (local address 127.1.245.126:34839)
0806 18:45:26.479796 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:26.479800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:26.479921 (+ 121us) server_negotiation.cc:407] Connection header received
0806 18:45:26.480062 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:26.480065 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:26.480116 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:26.480178 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:26.480635 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.481160 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.481946 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:26.482171 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:26.482265 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:26.484971 (+ 2706us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:41289: BlockingRecv error: recv got EOF from 127.1.245.67:41289 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 18:45:27 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:27 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:27.114380 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:27.108508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:35573)
0806 18:45:27.108836 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:27.108851 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:27.108948 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:27.109181 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:27.109183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:27.109403 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:27.109502 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:27.109509 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.110281 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.110284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:27.110912 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:27.110922 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.111426 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.111430 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:27.111511 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:27.112083 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:27.112096 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:27.114201 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":78,"threads_started":1}
W20260806 18:45:27.114578 14941 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:27.108615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:35573 (local address 127.1.245.126:34839)
0806 18:45:27.108900 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:27.108904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:27.108918 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:27.109028 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:27.109030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:27.109074 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:27.109157 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:27.109605 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.110170 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.111029 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.111307 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.111405 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:27.114439 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:35573: BlockingRecv error: recv got EOF from 127.1.245.65:35573 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":97,"threads_started":1}
Aug 06 18:45:27 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:27 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:27.300364 14942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:27.294601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:52227)
0806 18:45:27.295006 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:27.295018 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:27.295125 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:27.295367 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:27.295370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:27.295571 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:27.295658 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:27.295664 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.296461 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.296464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:27.297051 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:27.297059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.297487 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.297490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:27.297587 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:27.298085 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:27.298096 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:27.300205 (+ 2109us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":92,"threads_started":1}
W20260806 18:45:27.300534 14941 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:27.294695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:52227 (local address 127.1.245.126:34839)
0806 18:45:27.294856 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:27.294861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:27.295091 (+ 230us) server_negotiation.cc:407] Connection header received
0806 18:45:27.295207 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:27.295210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:27.295260 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:27.295337 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:27.295771 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.296343 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.297157 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.297385 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.297471 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:27.300427 (+ 2956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:52227: BlockingRecv error: recv got EOF from 127.1.245.66:52227 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 06 18:45:27 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:27 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:27.491443 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:27.485691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:53379)
0806 18:45:27.486064 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:27.486078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:27.486190 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:27.486528 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:27.486531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:27.486735 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:27.486843 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:27.486849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.487596 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.487600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:27.488247 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:27.488258 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.488693 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.488697 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:27.488790 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:27.489362 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:27.489374 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:27.491318 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":91,"threads_started":1}
W20260806 18:45:27.491670 14941 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:27.485834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:53379 (local address 127.1.245.126:34839)
0806 18:45:27.486000 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:27.486013 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:27.486143 (+ 130us) server_negotiation.cc:407] Connection header received
0806 18:45:27.486354 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:27.486357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:27.486407 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:27.486478 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:27.486950 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.487490 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.488360 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:27.488585 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:27.488677 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:27.491545 (+ 2868us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:53379: BlockingRecv error: recv got EOF from 127.1.245.67:53379 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 06 18:45:28 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:28 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:28.120934 14944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:28.115151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:38871)
0806 18:45:28.115482 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:28.115496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:28.115594 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:28.115856 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:28.115859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:28.116072 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:28.116177 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:28.116183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.117215 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.117220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:28.117927 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:28.117941 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.118462 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.118465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:28.118530 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:28.118983 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:28.118994 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:28.120772 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":116,"threads_started":1}
W20260806 18:45:28.121031 14945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:28.115345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:38871 (local address 127.1.245.126:34839)
0806 18:45:28.115600 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:28.115603 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:28.115618 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:45:28.115659 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:28.115661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:28.115705 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:28.115783 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:28.116434 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.117041 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.118063 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.118351 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.118446 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:28.120914 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:38871: BlockingRecv error: recv got EOF from 127.1.245.65:38871 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
Aug 06 18:45:28 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:28 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:28.307453 14947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:28.301347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:44237)
0806 18:45:28.301666 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:28.301684 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:28.301827 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:28.302117 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:28.302120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:28.302368 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:28.302489 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:28.302496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.303416 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.303420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:28.304021 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:28.304030 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.304514 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.304519 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:28.304608 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:28.305161 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:28.305174 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:28.307292 (+ 2118us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":93,"threads_started":1}
W20260806 18:45:28.307730 14945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:28.301476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:44237 (local address 127.1.245.126:34839)
0806 18:45:28.301668 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:28.301673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:28.301787 (+ 114us) server_negotiation.cc:407] Connection header received
0806 18:45:28.301926 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:28.301929 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:28.301995 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:28.302069 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:28.302620 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.303299 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.304155 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.304390 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.304485 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:28.307483 (+ 2998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:44237: BlockingRecv error: recv got EOF from 127.1.245.66:44237 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
Aug 06 18:45:28 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:28 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:28.497915 14948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:28.492323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:49373)
0806 18:45:28.492582 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:28.492597 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:28.492697 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:28.492952 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:28.492954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:28.493135 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:28.493234 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:28.493240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.494009 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.494012 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:28.494701 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:28.494714 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.495183 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.495186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:28.495315 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:28.495782 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:28.495795 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:28.497738 (+ 1943us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":77,"threads_started":1}
W20260806 18:45:28.498018 14945 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:28.492419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:49373 (local address 127.1.245.126:34839)
0806 18:45:28.492553 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:28.492557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:28.492660 (+ 103us) server_negotiation.cc:407] Connection header received
0806 18:45:28.492775 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:28.492778 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:28.492857 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:28.492929 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:28.493337 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.493877 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.494833 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:28.495057 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:28.495140 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:28.497892 (+ 2752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:49373: BlockingRecv error: recv got EOF from 127.1.245.67:49373 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 06 18:45:29 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:29 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:29.132884 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:29.121825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:44689)
0806 18:45:29.122146 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:29.122161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:29.122263 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:29.122565 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:29.122568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:29.122751 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:29.122854 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:29.122861 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.123575 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.123577 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:29.124118 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:29.124124 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.124601 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.124604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:29.124667 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:29.130694 (+ 6027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:29.130709 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:29.132740 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":82,"threads_started":1}
W20260806 18:45:29.133046 14952 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:29.121937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:44689 (local address 127.1.245.126:34839)
0806 18:45:29.122208 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:29.122212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:29.122226 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:29.122359 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:29.122362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:29.122406 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:29.122484 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:29.122972 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.123497 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.124236 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.124500 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.124582 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:29.132903 (+ 8321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:44689: BlockingRecv error: recv got EOF from 127.1.245.65:44689 (error 108)
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
Aug 06 18:45:29 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:29 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:29.314467 14954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:29.308234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:47407)
0806 18:45:29.308545 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:29.308560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:29.308683 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:29.308949 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:29.308952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:29.309162 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:29.309269 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:29.309277 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.310034 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.310038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:29.310627 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:29.310636 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.311087 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.311091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:29.311202 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:29.311904 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:29.311919 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:29.314284 (+ 2365us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":95,"threads_started":1}
W20260806 18:45:29.314589 14952 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:29.308352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:47407 (local address 127.1.245.126:34839)
0806 18:45:29.308525 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:29.308530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:29.308634 (+ 104us) server_negotiation.cc:407] Connection header received
0806 18:45:29.308782 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:29.308786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:29.308838 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:29.308904 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:29.309398 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.309924 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.310740 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.310970 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.311071 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:29.314487 (+ 3416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:47407: BlockingRecv error: recv got EOF from 127.1.245.66:47407 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 18:45:29 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:29 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:29.504648 14955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:29.498838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:49437)
0806 18:45:29.499125 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:29.499140 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:29.499252 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:29.499496 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:29.499498 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:29.499681 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:29.499780 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:29.499785 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.500604 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.500607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:29.501168 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:29.501176 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.501666 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.501669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:29.501739 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:29.502304 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:29.502355 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:29.504502 (+ 2147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":82,"threads_started":1}
W20260806 18:45:29.504767 14952 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:29.498953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:49437 (local address 127.1.245.126:34839)
0806 18:45:29.499108 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:29.499113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:29.499225 (+ 112us) server_negotiation.cc:407] Connection header received
0806 18:45:29.499341 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:29.499344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:29.499397 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:29.499468 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:29.499904 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.500473 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.501319 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:29.501553 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:29.501637 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:29.504646 (+ 3009us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:49437: BlockingRecv error: recv got EOF from 127.1.245.67:49437 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 18:45:30 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:30 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:30.139370 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:30.133798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:56097)
0806 18:45:30.134111 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:30.134125 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:30.134227 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:30.134497 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:30.134498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:30.134680 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:30.134784 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:30.134790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.135568 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.135571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:30.136174 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:30.136184 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.136712 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.136715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:30.136805 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:30.137304 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:30.137315 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:30.139183 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":94,"threads_started":1}
W20260806 18:45:30.139505 14958 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:30.133891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:56097 (local address 127.1.245.126:34839)
0806 18:45:30.134119 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:30.134122 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:30.134196 (+ 74us) server_negotiation.cc:407] Connection header received
0806 18:45:30.134344 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:30.134346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:30.134392 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:30.134468 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:30.134898 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.135441 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.136337 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.136569 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.136665 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:30.139374 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:56097: BlockingRecv error: recv got EOF from 127.1.245.65:56097 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":62,"threads_started":1}
Aug 06 18:45:30 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:30 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:30.321287 14959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:30.315684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:36093)
0806 18:45:30.315974 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:30.315989 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:30.316104 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:30.316375 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:30.316379 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:30.316590 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:30.316688 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:30.316694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.317489 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.317493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:30.318117 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:30.318128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.318647 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.318650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:30.318731 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:30.319269 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:30.319283 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:30.321133 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":92,"threads_started":1}
W20260806 18:45:30.321403 14958 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:30.315814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:36093 (local address 127.1.245.126:34839)
0806 18:45:30.315957 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:30.315961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:30.316073 (+ 112us) server_negotiation.cc:407] Connection header received
0806 18:45:30.316207 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:30.316211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:30.316260 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:30.316369 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:30.316802 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.317377 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.318242 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.318535 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.318636 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:30.321290 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:36093: BlockingRecv error: recv got EOF from 127.1.245.66:36093 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 06 18:45:30 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:30 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:30.511384 14960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:30.505551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:42535)
0806 18:45:30.505888 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:30.505906 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:30.506049 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:30.506354 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:30.506357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:30.506566 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:30.506667 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:30.506674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.507502 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.507506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:30.508320 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:30.508333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.508804 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.508809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:30.508885 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:30.509411 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:30.509423 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:30.511259 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":105,"threads_started":1}
W20260806 18:45:30.511593 14958 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:30.505757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:42535 (local address 127.1.245.126:34839)
0806 18:45:30.505931 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:30.505936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:30.505986 (+ 50us) server_negotiation.cc:407] Connection header received
0806 18:45:30.506142 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:30.506145 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:30.506205 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:30.506347 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:30.506788 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.507392 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.508466 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:30.508695 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:30.508814 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:30.511468 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:42535: BlockingRecv error: recv got EOF from 127.1.245.67:42535 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 06 18:45:31 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:31 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:31.145357 14961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:31.140151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:41877)
0806 18:45:31.140446 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:31.140460 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:31.140559 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:31.140763 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:31.140765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:31.140946 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:31.141045 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:31.141050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.141798 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.141801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:31.142446 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:31.142458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.142898 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.142900 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:31.142959 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:31.143422 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:31.143433 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:31.145179 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":80,"threads_started":1}
W20260806 18:45:31.145440 14962 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:31.140254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:41877 (local address 127.1.245.126:34839)
0806 18:45:31.140530 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:31.140534 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:31.140547 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:45:31.140629 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:31.140632 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:31.140674 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:31.140751 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:31.141140 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.141676 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.142559 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.142807 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.142889 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:31.145320 (+ 2431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:41877: BlockingRecv error: recv got EOF from 127.1.245.65:41877 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":102,"threads_started":1}
Aug 06 18:45:31 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:31 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:31.327498 14963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:31.322195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:44111)
0806 18:45:31.322519 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:31.322534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:31.322656 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:31.322937 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:31.322940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:31.323135 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:31.323231 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:31.323237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.324003 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.324006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:31.324563 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:31.324572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.325006 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.325010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:31.325095 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:31.325520 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:31.325531 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:31.327370 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":141,"threads_started":1}
W20260806 18:45:31.327658 14962 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:31.322308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:44111 (local address 127.1.245.126:34839)
0806 18:45:31.322487 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:31.322491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:31.322606 (+ 115us) server_negotiation.cc:407] Connection header received
0806 18:45:31.322760 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:31.322764 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:31.322815 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:31.322877 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:31.323346 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.323893 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.324670 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.324903 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.324991 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:31.327532 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:44111: BlockingRecv error: recv got EOF from 127.1.245.66:44111 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 18:45:31 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:31 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:31.517660 14964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:31.512186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:35303)
0806 18:45:31.512489 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:31.512504 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:31.512627 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:31.512940 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:31.512943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:31.513158 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:31.513331 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:31.513340 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.514122 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.514125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:31.514758 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:31.514768 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.515192 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.515194 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:31.515254 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:31.515790 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:31.515803 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:31.517536 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":122,"threads_started":1}
W20260806 18:45:31.517946 14962 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:31.512239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:35303 (local address 127.1.245.126:34839)
0806 18:45:31.512461 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:31.512465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:31.512576 (+ 111us) server_negotiation.cc:407] Connection header received
0806 18:45:31.512722 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:31.512724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:31.512787 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:31.512868 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:31.513468 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.514020 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.514876 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:31.515105 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:31.515187 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:31.517827 (+ 2640us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:35303: BlockingRecv error: recv got EOF from 127.1.245.67:35303 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
Aug 06 18:45:32 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:32 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:32.151979 14965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:32.146182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:36281)
0806 18:45:32.146550 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:32.146569 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:32.146692 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:32.147009 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:32.147012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:32.147213 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:32.147337 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:32.147344 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.148201 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.148206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:32.148901 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:32.148913 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.149469 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.149472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:32.149547 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:32.150045 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:32.150058 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:32.151860 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":82,"threads_started":1}
W20260806 18:45:32.152146 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:32.146431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:36281 (local address 127.1.245.126:34839)
0806 18:45:32.146750 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:32.146753 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:32.146767 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:32.146807 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:32.146809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:32.146852 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:32.146935 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:32.147444 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.148055 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.149028 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.149336 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.149427 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:32.152013 (+ 2586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:36281: BlockingRecv error: recv got EOF from 127.1.245.65:36281 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":138,"threads_started":1}
Aug 06 18:45:32 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:32 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:32.333657 14967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:32.328268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:55127)
0806 18:45:32.328586 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:32.328601 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:32.328734 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:32.328998 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:32.329001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:32.329206 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:32.329343 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:32.329349 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.330100 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.330103 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:32.330719 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:32.330729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.331151 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.331155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:32.331239 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:32.331737 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:32.331749 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:32.333503 (+ 1754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":100,"threads_started":1}
W20260806 18:45:32.333756 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:32.328391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:55127 (local address 127.1.245.126:34839)
0806 18:45:32.328568 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:32.328572 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:32.328673 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:45:32.328837 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:32.328841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:32.328888 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:32.328955 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:32.329463 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.329991 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.330831 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.331049 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.331137 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:32.333651 (+ 2514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:55127: BlockingRecv error: recv got EOF from 127.1.245.66:55127 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 06 18:45:32 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:32 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:32.523610 14968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:32.518405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:44321)
0806 18:45:32.518661 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:32.518674 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:32.518772 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:32.518990 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:32.518992 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:32.519167 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:32.519269 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:32.519274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.520010 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.520012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:32.520564 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:32.520572 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.521036 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.521039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:32.521106 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:32.521637 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:32.521650 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:32.523490 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":75,"threads_started":1}
W20260806 18:45:32.523759 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:32.518461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:44321 (local address 127.1.245.126:34839)
0806 18:45:32.518614 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:32.518618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:32.518725 (+ 107us) server_negotiation.cc:407] Connection header received
0806 18:45:32.518837 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:32.518839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:32.518886 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:32.518952 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:32.519383 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.519927 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.520676 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:32.520922 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:32.521004 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:32.523636 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:44321: BlockingRecv error: recv got EOF from 127.1.245.67:44321 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 06 18:45:33 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:33 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:33.158285 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:33.152847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:53391)
0806 18:45:33.153112 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:33.153127 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:33.153220 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:33.153437 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:33.153439 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:33.153620 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:33.153715 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:33.153721 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.154556 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.154559 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:33.155139 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:33.155148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.155611 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.155613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:33.155671 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:33.156142 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:33.156155 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:33.158130 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
W20260806 18:45:33.158550 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:33.152946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:53391 (local address 127.1.245.126:34839)
0806 18:45:33.153208 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:33.153212 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:33.153226 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:33.153294 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:33.153297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:33.153340 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:33.153428 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:33.153861 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.154452 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.155276 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.155511 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.155597 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:33.158389 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:53391: BlockingRecv error: recv got EOF from 127.1.245.65:53391 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":74,"threads_started":1}
Aug 06 18:45:33 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:33 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:33.339852 14971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:33.334500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:50061)
0806 18:45:33.334779 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:33.334794 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:33.334930 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:33.335171 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:33.335174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:33.335407 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:33.335515 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:33.335522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.336338 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.336342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:33.336942 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:33.336951 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.337374 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.337377 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:33.337453 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:33.337927 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:33.337938 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:33.339708 (+ 1770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":81,"threads_started":1}
W20260806 18:45:33.339973 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:33.334644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:50061 (local address 127.1.245.126:34839)
0806 18:45:33.334780 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:33.334785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:33.334857 (+ 72us) server_negotiation.cc:407] Connection header received
0806 18:45:33.335017 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:33.335020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:33.335071 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:33.335136 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:33.335620 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.336219 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.337051 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.337275 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.337361 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:33.339854 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:50061: BlockingRecv error: recv got EOF from 127.1.245.66:50061 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 18:45:33 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:33 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:33.529757 14974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:33.524342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:49815)
0806 18:45:33.524655 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:33.524670 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:33.524768 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:33.525032 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:33.525034 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:33.525218 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:33.525346 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:33.525352 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.526112 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.526116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:33.526762 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:33.526778 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.527245 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.527248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:33.527333 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:33.527835 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:33.527847 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:33.529624 (+ 1777us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":109,"threads_started":1}
W20260806 18:45:33.529872 14970 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:33.524450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:49815 (local address 127.1.245.126:34839)
0806 18:45:33.524603 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:33.524608 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:33.524730 (+ 122us) server_negotiation.cc:407] Connection header received
0806 18:45:33.524872 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:33.524875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:33.524929 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:33.525012 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:33.525443 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.525993 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.526882 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:33.527107 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:33.527215 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:33.529756 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:49815: BlockingRecv error: recv got EOF from 127.1.245.67:49815 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 06 18:45:34 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:34 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:34.168649 14975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:34.159123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:43073)
0806 18:45:34.159473 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:34.159491 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:34.159610 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:34.159910 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:34.159913 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:34.160146 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:34.160247 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:34.160253 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.161032 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.161035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:34.161593 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:34.161602 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.162042 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.162046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:34.162131 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:34.166411 (+ 4280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:34.166428 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:34.168519 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":91,"threads_started":1}
W20260806 18:45:34.168885 14976 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:34.159199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:43073 (local address 127.1.245.126:34839)
0806 18:45:34.159525 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:34.159529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:34.159544 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:45:34.159723 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:34.159726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:34.159776 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:34.159876 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:34.160374 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.160920 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.161708 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.161939 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.162026 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:34.168719 (+ 6693us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:43073: BlockingRecv error: recv got EOF from 127.1.245.65:43073 (error 108)
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":83,"threads_started":1}
Aug 06 18:45:34 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:34 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:34.345793 14977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:34.340658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:35613)
0806 18:45:34.340906 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:34.340919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:34.341023 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:34.341241 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:34.341243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:34.341420 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:34.341534 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:34.341540 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.342224 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.342226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:34.342862 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:34.342872 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.343271 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.343274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:34.343329 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:34.343839 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:34.343850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:34.345672 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":89,"threads_started":1}
W20260806 18:45:34.345961 14976 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:34.340675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:35613 (local address 127.1.245.126:34839)
0806 18:45:34.340827 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:34.340831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:34.341018 (+ 187us) server_negotiation.cc:407] Connection header received
0806 18:45:34.341104 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:34.341106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:34.341156 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:34.341231 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:34.341624 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.342147 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.342968 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.343188 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.343265 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:34.345834 (+ 2569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:35613: BlockingRecv error: recv got EOF from 127.1.245.66:35613 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 06 18:45:34 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:34 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.67@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:34.536194 14978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:34.530522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.67:60671)
0806 18:45:34.530834 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:34.530850 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:34.530973 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:34.531223 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:34.531226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:34.531461 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:34.531560 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:34.531566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.532387 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.532391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:34.532974 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:34.532983 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.533431 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.533435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:34.533517 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:34.534032 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:34.534045 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:34.536055 (+ 2010us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":91,"threads_started":1}
W20260806 18:45:34.536343 14976 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:34.530627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:60671 (local address 127.1.245.126:34839)
0806 18:45:34.530816 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:34.530820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:34.530923 (+ 103us) server_negotiation.cc:407] Connection header received
0806 18:45:34.531061 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:34.531064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:34.531114 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:34.531180 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:34.531682 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.532272 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.533103 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:34.533326 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:34.533416 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:34.536217 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.67:60671: BlockingRecv error: recv got EOF from 127.1.245.67:60671 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 06 18:45:35 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:35 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.65@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:35.175401 14979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:35.169630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.65:50051)
0806 18:45:35.169952 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:35.169966 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:35.170056 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:35.170343 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:35.170346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:35.170538 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:35.170641 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:35.170647 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.171438 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.171441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:35.172117 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:35.172130 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.172714 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.172717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:35.172826 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:35.173362 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:35.173375 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:35.175284 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":77,"threads_started":1}
W20260806 18:45:35.175566 14980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:35.169749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:50051 (local address 127.1.245.126:34839)
0806 18:45:35.170035 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:35.170038 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:35.170052 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:45:35.170146 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:35.170149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:35.170192 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:35.170284 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:35.170749 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.171310 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.172213 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.172596 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.172714 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:35.175434 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.65:50051: BlockingRecv error: recv got EOF from 127.1.245.65:50051 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
Aug 06 18:45:35 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 18:45:35 dist-test-slave-fgb3 krb5kdc[13572](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.245.66@KRBTEST.COM for oryx/127.1.245.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 18:45:35.352131 14982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:35.346710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:34839 (local address 127.1.245.66:36777)
0806 18:45:35.346978 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:35.346992 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:35.347090 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:35.347314 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:35.347316 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:35.347506 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:35.347601 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:35.347607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.348419 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.348422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:35.349084 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:35.349098 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.349624 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.349628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:35.349699 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:35.350161 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:35.350173 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:35.352011 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.245.126:34839: Server oryx/127.1.245.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":72,"threads_started":1}
W20260806 18:45:35.352280 14980 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:35.346797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:36777 (local address 127.1.245.126:34839)
0806 18:45:35.346944 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:35.346948 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:35.347054 (+ 106us) server_negotiation.cc:407] Connection header received
0806 18:45:35.347163 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:35.347166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:35.347212 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:35.347291 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:35.347691 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.348277 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.349203 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.349504 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.349606 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:35.352154 (+ 2548us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.245.66:36777: BlockingRecv error: recv got EOF from 127.1.245.66:36777 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:45:35.428733 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 14341
I20260806 18:45:35.446060 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 14561
I20260806 18:45:35.465641 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 14732
I20260806 18:45:35.491410 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 14242
2026-08-06T18:45:35Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18153 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T18:45:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:35Z Disabled control of system clock
I20260806 18:45:35.530427 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:44363
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:44363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 18:45:35.641968 14987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:35.642225 14987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:35.642258 14987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:35.646179 14987 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:35.646291 14987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:35.646307 14987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:35.646341 14987 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:35.646354 14987 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:35.651137 14987 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:44363
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:44363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:35.652277 14987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:35.653687 14987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:35.662957 14994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:35.663024 14997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:35.663024 14992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:35.663159 14987 server_base.cc:1034] running on GCE node
I20260806 18:45:35.663872 14987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:35.665091 14987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:35.666345 14987 hybrid_clock.cc:648] HybridClock initialized: now 1786041935666296 us; error 35 us; skew 500 ppm
I20260806 18:45:35.668381 14987 webserver.cc:460] Webserver started at http://127.1.245.126:34425/ using document root <none> and password file <none>
I20260806 18:45:35.668978 14987 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:35.669052 14987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:35.669317 14987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:35.671535 14987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "1fd31235c5394a9599597a94719a30f3"
format_stamp: "Formatted at 2026-08-06 18:45:35 on dist-test-slave-fgb3"
server_key: "5282f00c27a2e7c608cf0aa00543a8c3"
server_key_iv: "a8fbcf80e7cf0016135cc8792a4b9859"
server_key_version: "encryptionkey@0"
I20260806 18:45:35.672258 14987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "1fd31235c5394a9599597a94719a30f3"
format_stamp: "Formatted at 2026-08-06 18:45:35 on dist-test-slave-fgb3"
server_key: "5282f00c27a2e7c608cf0aa00543a8c3"
server_key_iv: "a8fbcf80e7cf0016135cc8792a4b9859"
server_key_version: "encryptionkey@0"
I20260806 18:45:35.676385 14987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:45:35.679046 15006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:35.680366 14987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:35.680505 14987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "1fd31235c5394a9599597a94719a30f3"
format_stamp: "Formatted at 2026-08-06 18:45:35 on dist-test-slave-fgb3"
server_key: "5282f00c27a2e7c608cf0aa00543a8c3"
server_key_iv: "a8fbcf80e7cf0016135cc8792a4b9859"
server_key_version: "encryptionkey@0"
I20260806 18:45:35.680611 14987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:35.692205 14987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:35.693018 14987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:35.693234 14987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:35.701205 14987 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:44363
I20260806 18:45:35.701262 15078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:44363 every 8 connection(s)
I20260806 18:45:35.702248 14987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:35.705788 15079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:35.706496 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 14987
I20260806 18:45:35.706597 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:35.706880 2005 external_mini_cluster.cc:1428] Setting key 78a8da260d88cdec22e5208a2f6982e9
I20260806 18:45:35.712224 15079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3: Bootstrap starting.
I20260806 18:45:35.714810 15079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:35.715629 15079 log.cc:824] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:35.716001 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:35.708244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.1.245.126:44363)
0806 18:45:35.708886 (+ 642us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:35.708903 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:35.708941 (+ 38us) server_negotiation.cc:407] Connection header received
0806 18:45:35.709606 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:35.709624 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:35.710026 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:35.710309 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:35.710694 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.711641 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.712749 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.713251 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.713378 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:35.714228 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:35.714234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:35.714247 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:35.714664 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:35.714721 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:35.714742 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:35.714822 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:35.714982 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:45:35.715255 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":124,"threads_started":1}
I20260806 18:45:35.717666 15079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3: No bootstrap required, opened a new log
I20260806 18:45:35.721359 15079 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd31235c5394a9599597a94719a30f3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44363 } }
I20260806 18:45:35.721596 15079 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:35.721644 15079 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fd31235c5394a9599597a94719a30f3, State: Initialized, Role: FOLLOWER
I20260806 18:45:35.722009 15079 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd31235c5394a9599597a94719a30f3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44363 } }
I20260806 18:45:35.722119 15079 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:35.722162 15079 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:35.722230 15079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:35.723124 15079 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd31235c5394a9599597a94719a30f3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44363 } }
I20260806 18:45:35.723421 15079 leader_election.cc:302] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fd31235c5394a9599597a94719a30f3; no voters:
I20260806 18:45:35.723708 15079 leader_election.cc:288] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:35.723834 15084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:35.724048 15084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [term 1 LEADER]: Becoming Leader. State: Replica: 1fd31235c5394a9599597a94719a30f3, State: Running, Role: LEADER
I20260806 18:45:35.724401 15084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd31235c5394a9599597a94719a30f3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44363 } }
I20260806 18:45:35.724979 15079 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:35.726024 15087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fd31235c5394a9599597a94719a30f3. Latest consensus state: current_term: 1 leader_uuid: "1fd31235c5394a9599597a94719a30f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd31235c5394a9599597a94719a30f3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44363 } } }
I20260806 18:45:35.726050 15086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fd31235c5394a9599597a94719a30f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fd31235c5394a9599597a94719a30f3" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 44363 } } }
I20260806 18:45:35.726199 15087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:35.726199 15086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:35.726722 15096 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:35.730036 15096 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:35.735685 15096 catalog_manager.cc:1347] Generated new cluster ID: 2e400927974544518b33b58fe73384c8
I20260806 18:45:35.735754 15096 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:35.759210 15096 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:35.760434 15096 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:35.770684 15096 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fd31235c5394a9599597a94719a30f3: Generated new TSK 0
I20260806 18:45:35.771624 15096 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:45:35.782409 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:44363
--builtin_ntp_servers=127.1.245.84:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 18:45:35.901361 15112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:35.901628 15112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:35.901665 15112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:35.905637 15112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:35.905699 15112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:35.905751 15112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:35.910470 15112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:35.911548 15112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:35.912889 15112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:35.922684 15120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:35.922711 15118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:35.922711 15122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:35.922802 15112 server_base.cc:1034] running on GCE node
I20260806 18:45:35.923408 15112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:35.924062 15112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:35.925225 15112 hybrid_clock.cc:648] HybridClock initialized: now 1786041935925231 us; error 63 us; skew 500 ppm
I20260806 18:45:35.927340 15112 webserver.cc:460] Webserver started at http://127.1.245.65:35765/ using document root <none> and password file <none>
I20260806 18:45:35.927973 15112 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:35.928042 15112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:35.928251 15112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:35.930392 15112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "bdc2721e9cfd40bf8b884e51f362b2a0"
format_stamp: "Formatted at 2026-08-06 18:45:35 on dist-test-slave-fgb3"
server_key: "9c33ad059e5d4d5e1e3afe6d5b508e17"
server_key_iv: "80d280e6141bfc23cb5bd5bc336e0cca"
server_key_version: "encryptionkey@0"
I20260806 18:45:35.930953 15112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "bdc2721e9cfd40bf8b884e51f362b2a0"
format_stamp: "Formatted at 2026-08-06 18:45:35 on dist-test-slave-fgb3"
server_key: "9c33ad059e5d4d5e1e3afe6d5b508e17"
server_key_iv: "80d280e6141bfc23cb5bd5bc336e0cca"
server_key_version: "encryptionkey@0"
I20260806 18:45:35.934720 15112 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260806 18:45:35.937237 15132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:35.938616 15112 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:35.938727 15112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "bdc2721e9cfd40bf8b884e51f362b2a0"
format_stamp: "Formatted at 2026-08-06 18:45:35 on dist-test-slave-fgb3"
server_key: "9c33ad059e5d4d5e1e3afe6d5b508e17"
server_key_iv: "80d280e6141bfc23cb5bd5bc336e0cca"
server_key_version: "encryptionkey@0"
I20260806 18:45:35.938844 15112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:35.958834 15112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:35.959574 15112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:35.959757 15112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:35.960381 15112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:35.961380 15112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:35.961431 15112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:35.961470 15112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:35.961498 15112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:35.969913 15112 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:40629
I20260806 18:45:35.970012 15289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:40629 every 8 connection(s)
I20260806 18:45:35.970916 15112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:35.980321 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:35.972778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:52551 (local address 127.1.245.126:44363)
0806 18:45:35.972931 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:35.972936 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:35.974033 (+ 1097us) server_negotiation.cc:407] Connection header received
0806 18:45:35.975457 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:35.975460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:35.975508 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:35.975581 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:35.976450 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.976981 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.978275 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.978572 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.978691 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:35.979781 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:35.979784 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:35.979787 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:35.979801 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:35.979821 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:35.979828 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:35.979890 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:35.980078 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:45:35.980146 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:45:35.980340 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 15112
I20260806 18:45:35.980432 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:45:35.980701 2005 external_mini_cluster.cc:1428] Setting key b619872fb47767743410d447717aa43d
I20260806 18:45:35.981156 15296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:35.973185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:44363 (local address 127.1.245.65:52551)
0806 18:45:35.973889 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:35.973930 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:35.975247 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:35.975750 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:35.975757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:35.975839 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:35.976317 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:35.976329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.977102 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.977105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:35.978099 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:35.978109 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:35.978761 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:35.978764 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:35.978936 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:35.979513 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:35.979515 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:35.979554 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:35.979618 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:35.979930 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:35.979939 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:35.979944 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:45:35.980060 (+ 116us) client_negotiation.cc:240] Negotiation successful
0806 18:45:35.980288 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":116,"threads_started":1}
I20260806 18:45:35.982362 15290 heartbeater.cc:347] Connected to a master server at 127.1.245.126:44363
I20260806 18:45:35.982592 15290 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:35.982785 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:44363
--builtin_ntp_servers=127.1.245.84:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:45:35.983134 15290 heartbeater.cc:511] Master 127.1.245.126:44363 requested a full tablet report, sending...
I20260806 18:45:35.984591 15029 ts_manager.cc:194] Registered new tserver with Master: bdc2721e9cfd40bf8b884e51f362b2a0 (127.1.245.65:40629)
I20260806 18:45:35.985767 15029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.65:52551
W20260806 18:45:36.093807 15297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:36.094069 15297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:36.094105 15297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:36.098124 15297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:36.098181 15297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:36.098234 15297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:36.102928 15297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:36.103988 15297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:36.105361 15297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:36.114703 15306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:36.114847 15304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:36.115099 15297 server_base.cc:1034] running on GCE node
W20260806 18:45:36.115263 15303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:36.115669 15297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:36.116309 15297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:36.117483 15297 hybrid_clock.cc:648] HybridClock initialized: now 1786041936117452 us; error 42 us; skew 500 ppm
I20260806 18:45:36.119341 15297 webserver.cc:460] Webserver started at http://127.1.245.66:35311/ using document root <none> and password file <none>
I20260806 18:45:36.119891 15297 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:36.119961 15297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:36.120163 15297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:36.122119 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "e766e5afe3bd4eb98b0df63e57722a14"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "982384ab1c328619ba862314f7b6ebe9"
server_key_iv: "4932267909c289f30215cd051cd0b657"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.122685 15297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "e766e5afe3bd4eb98b0df63e57722a14"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "982384ab1c328619ba862314f7b6ebe9"
server_key_iv: "4932267909c289f30215cd051cd0b657"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.126222 15297 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:45:36.128969 15314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.130218 15297 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:36.130349 15297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "e766e5afe3bd4eb98b0df63e57722a14"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "982384ab1c328619ba862314f7b6ebe9"
server_key_iv: "4932267909c289f30215cd051cd0b657"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.130451 15297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:36.143473 15297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:36.144205 15297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:36.144381 15297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:36.144953 15297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:36.145888 15297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:36.145936 15297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.145987 15297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:36.146016 15297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.154575 15297 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:43313
I20260806 18:45:36.154599 15468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:43313 every 8 connection(s)
I20260806 18:45:36.155498 15297 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:36.159615 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 15297
I20260806 18:45:36.159739 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:45:36.160036 2005 external_mini_cluster.cc:1428] Setting key b209ae813618ac3390ac093edd9cc1c3
I20260806 18:45:36.162119 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:44363
--builtin_ntp_servers=127.1.245.84:42807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 18:45:36.165392 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:36.157352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:46557 (local address 127.1.245.126:44363)
0806 18:45:36.157482 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:36.157486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:36.158611 (+ 1125us) server_negotiation.cc:407] Connection header received
0806 18:45:36.160314 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:36.160317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:36.160373 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:36.160483 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:36.161393 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:36.162110 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:36.163326 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:36.163623 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:36.163706 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:36.164941 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:36.164945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:36.164947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:36.164959 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:36.164980 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:36.164993 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:36.165048 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:36.165218 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 18:45:36.165269 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:45:36.166635 15471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:36.157752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:44363 (local address 127.1.245.66:46557)
0806 18:45:36.158475 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:36.158517 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:36.159965 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:36.160668 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:36.160676 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:36.160757 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:36.161251 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:36.161263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:36.162270 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:36.162275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:36.163193 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:36.163203 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:36.163795 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:36.163800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:36.164002 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:36.164637 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:36.164639 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:36.164694 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:36.164790 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:36.165076 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:36.165089 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:36.165095 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:45:36.165220 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 18:45:36.165519 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":135,"threads_started":1}
I20260806 18:45:36.167778 15469 heartbeater.cc:347] Connected to a master server at 127.1.245.126:44363
I20260806 18:45:36.168012 15469 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:36.168561 15469 heartbeater.cc:511] Master 127.1.245.126:44363 requested a full tablet report, sending...
I20260806 18:45:36.169610 15029 ts_manager.cc:194] Registered new tserver with Master: e766e5afe3bd4eb98b0df63e57722a14 (127.1.245.66:43313)
I20260806 18:45:36.170413 15029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.66:46557
W20260806 18:45:36.275867 15472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:36.276139 15472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:36.276175 15472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:36.280303 15472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:36.280360 15472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:36.280413 15472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:36.285292 15472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:44363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:36.286429 15472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:36.287835 15472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:36.297534 15480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:36.297598 15483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:36.297590 15472 server_base.cc:1034] running on GCE node
W20260806 18:45:36.297793 15481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:36.298259 15472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:36.298930 15472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:36.300089 15472 hybrid_clock.cc:648] HybridClock initialized: now 1786041936300072 us; error 27 us; skew 500 ppm
I20260806 18:45:36.301944 15472 webserver.cc:460] Webserver started at http://127.1.245.67:37725/ using document root <none> and password file <none>
I20260806 18:45:36.302577 15472 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:36.302644 15472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:36.302820 15472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:36.304819 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "7ed5acc8f0544b3b8af1720dbe9a3a0d"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "b682ac8c351101e112416a78e89f92f0"
server_key_iv: "7abac270749fe1316a608c43f21ba4e7"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.305328 15472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "7ed5acc8f0544b3b8af1720dbe9a3a0d"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "b682ac8c351101e112416a78e89f92f0"
server_key_iv: "7abac270749fe1316a608c43f21ba4e7"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.308923 15472 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:45:36.311514 15489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.312743 15472 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:36.312844 15472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "7ed5acc8f0544b3b8af1720dbe9a3a0d"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "b682ac8c351101e112416a78e89f92f0"
server_key_iv: "7abac270749fe1316a608c43f21ba4e7"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.312939 15472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:36.331317 15472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:36.332016 15472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:36.332181 15472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:36.332767 15472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:36.333745 15472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:36.333791 15472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.333833 15472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:36.333861 15472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.342336 15472 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:33073
I20260806 18:45:36.342383 15651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:33073 every 8 connection(s)
I20260806 18:45:36.343251 15472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:36.349428 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 15472
I20260806 18:45:36.349550 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:36.349800 2005 external_mini_cluster.cc:1428] Setting key 9ca886a61f3b2bcb386b4052c2b5b8da
I20260806 18:45:36.352742 15082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:36.345088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:35767 (local address 127.1.245.126:44363)
0806 18:45:36.345217 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:36.345220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:36.346341 (+ 1121us) server_negotiation.cc:407] Connection header received
0806 18:45:36.347723 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:36.347735 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:36.347796 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:36.347870 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:36.348777 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:36.349403 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:36.350823 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:36.351127 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:36.351228 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:36.352374 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:36.352377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:36.352380 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:36.352391 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:36.352411 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:36.352419 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:36.352482 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:36.352588 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 18:45:36.352636 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:45:36.353572 15654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:36.345439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:44363 (local address 127.1.245.67:35767)
0806 18:45:36.346114 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:36.346154 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:36.347508 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:36.348061 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:36.348068 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:36.348154 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:36.348628 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:36.348640 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:36.349550 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:36.349554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:36.350703 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:36.350714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:36.351276 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:36.351280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:36.351466 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:36.352117 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:36.352118 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:36.352157 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:36.352218 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:36.352487 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:36.352492 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:36.352495 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:45:36.352578 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:45:36.352787 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":111,"threads_started":1}
I20260806 18:45:36.354632 15652 heartbeater.cc:347] Connected to a master server at 127.1.245.126:44363
I20260806 18:45:36.354843 15652 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:36.355326 15652 heartbeater.cc:511] Master 127.1.245.126:44363 requested a full tablet report, sending...
I20260806 18:45:36.356314 15029 ts_manager.cc:194] Registered new tserver with Master: 7ed5acc8f0544b3b8af1720dbe9a3a0d (127.1.245.67:33073)
I20260806 18:45:36.356843 15029 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.67:35767
I20260806 18:45:36.363586 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:45:36.366184 15082 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:36.365390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.1.245.126:44363)
0806 18:45:36.365529 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:36.365532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:36.365625 (+ 93us) server_negotiation.cc:407] Connection header received
0806 18:45:36.365721 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:36.365723 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:36.365761 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:36.365820 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:36.366072 (+ 252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41116: BlockingRecv error: recv got EOF from 127.0.0.1:41116 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:45:36.366562 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 15112
I20260806 18:45:36.378223 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 15297
I20260806 18:45:36.398124 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 15472
I20260806 18:45:36.408231 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 14987
2026-08-06T18:45:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (911 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T18:45:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:36Z Disabled control of system clock
I20260806 18:45:36.441931 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35285
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35285
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:45:36.554622 15666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:36.554878 15666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:36.554915 15666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:36.558933 15666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:36.558990 15666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:36.559003 15666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:36.559018 15666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:36.559032 15666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:36.563715 15666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35285
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35285
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:36.564795 15666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:36.566138 15666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:36.574400 15678 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:36.574362 15674 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:36.574402 15675 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:36.574594 15666 server_base.cc:1034] running on GCE node
I20260806 18:45:36.575201 15666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:36.576248 15666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:36.577431 15666 hybrid_clock.cc:648] HybridClock initialized: now 1786041936577408 us; error 32 us; skew 500 ppm
I20260806 18:45:36.579296 15666 webserver.cc:460] Webserver started at http://127.1.245.126:46019/ using document root <none> and password file <none>
I20260806 18:45:36.579857 15666 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:36.579929 15666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:36.580159 15666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:36.582201 15666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "de76abe05b244243aebadfb5ac5eb79b"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "cfda1537a7e226760e487f4e296b5343"
server_key_iv: "04397897362dbd91523a702e346d6f14"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.582824 15666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "de76abe05b244243aebadfb5ac5eb79b"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "cfda1537a7e226760e487f4e296b5343"
server_key_iv: "04397897362dbd91523a702e346d6f14"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.586544 15666 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:45:36.589016 15687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.590226 15666 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:36.590412 15666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "de76abe05b244243aebadfb5ac5eb79b"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "cfda1537a7e226760e487f4e296b5343"
server_key_iv: "04397897362dbd91523a702e346d6f14"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.590515 15666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:36.595059 15666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:36.595659 15666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:36.595821 15666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:36.602697 15666 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:35285
I20260806 18:45:36.602702 15760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:35285 every 8 connection(s)
I20260806 18:45:36.603641 15666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:36.606874 15761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:36.607966 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 15666
I20260806 18:45:36.608078 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:36.608352 2005 external_mini_cluster.cc:1428] Setting key e5f03f1d8dc80c5c2462556403417969
I20260806 18:45:36.612949 15761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b: Bootstrap starting.
I20260806 18:45:36.614014 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:36.609715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33594 (local address 127.1.245.126:35285)
0806 18:45:36.610517 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:36.610538 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:36.610574 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:45:36.611245 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:36.611263 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:36.611611 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:36.611923 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:36.612709 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:36.612713 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:36.612727 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:36.612740 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:36.612796 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:36.612813 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:36.612950 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:36.613018 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 18:45:36.613315 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":125,"threads_started":1}
I20260806 18:45:36.615363 15761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:36.616231 15761 log.cc:824] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:36.618279 15761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b: No bootstrap required, opened a new log
I20260806 18:45:36.621775 15761 raft_consensus.cc:348] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de76abe05b244243aebadfb5ac5eb79b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35285 } }
I20260806 18:45:36.622007 15761 raft_consensus.cc:374] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:36.622064 15761 raft_consensus.cc:729] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de76abe05b244243aebadfb5ac5eb79b, State: Initialized, Role: FOLLOWER
I20260806 18:45:36.622545 15761 consensus_queue.cc:260] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de76abe05b244243aebadfb5ac5eb79b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35285 } }
I20260806 18:45:36.622678 15761 raft_consensus.cc:388] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:36.622723 15761 raft_consensus.cc:482] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:36.622797 15761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:36.623668 15761 raft_consensus.cc:504] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de76abe05b244243aebadfb5ac5eb79b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35285 } }
I20260806 18:45:36.623970 15761 leader_election.cc:302] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de76abe05b244243aebadfb5ac5eb79b; no voters:
I20260806 18:45:36.624233 15761 leader_election.cc:288] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:36.624418 15767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:36.624650 15767 raft_consensus.cc:686] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [term 1 LEADER]: Becoming Leader. State: Replica: de76abe05b244243aebadfb5ac5eb79b, State: Running, Role: LEADER
I20260806 18:45:36.625000 15767 consensus_queue.cc:237] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de76abe05b244243aebadfb5ac5eb79b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35285 } }
I20260806 18:45:36.625488 15761 sys_catalog.cc:564] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:36.626612 15768 sys_catalog.cc:455] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de76abe05b244243aebadfb5ac5eb79b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de76abe05b244243aebadfb5ac5eb79b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35285 } } }
I20260806 18:45:36.626747 15768 sys_catalog.cc:458] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:36.627164 15778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:36.627102 15769 sys_catalog.cc:455] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [sys.catalog]: SysCatalogTable state changed. Reason: New leader de76abe05b244243aebadfb5ac5eb79b. Latest consensus state: current_term: 1 leader_uuid: "de76abe05b244243aebadfb5ac5eb79b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de76abe05b244243aebadfb5ac5eb79b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35285 } } }
I20260806 18:45:36.627254 15769 sys_catalog.cc:458] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:36.630514 15778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:36.636665 15778 catalog_manager.cc:1347] Generated new cluster ID: bb47a62bc0a542cb9d09fae5c1a01178
I20260806 18:45:36.636746 15778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:36.661249 15778 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:36.661619 15778 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:36.671217 15778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de76abe05b244243aebadfb5ac5eb79b: Generated new TSK 0
I20260806 18:45:36.672098 15778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:45:36.678118 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35285
--builtin_ntp_servers=127.1.245.84:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:45:36.793924 15792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:36.794216 15792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:36.794258 15792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:36.798377 15792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:36.798448 15792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:36.798511 15792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:36.803385 15792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:36.804584 15792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:36.805974 15792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:36.815541 15804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:36.815548 15798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:36.815666 15792 server_base.cc:1034] running on GCE node
W20260806 18:45:36.815730 15799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:36.816296 15792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:36.816913 15792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:36.818080 15792 hybrid_clock.cc:648] HybridClock initialized: now 1786041936818052 us; error 41 us; skew 500 ppm
I20260806 18:45:36.819873 15792 webserver.cc:460] Webserver started at http://127.1.245.65:36717/ using document root <none> and password file <none>
I20260806 18:45:36.820436 15792 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:36.820508 15792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:36.820698 15792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:36.822793 15792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "62407086a4504fef87c0f60140126905"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "103199b0f3a45d9d60fbc34554127414"
server_key_iv: "5c5b3aca2b63d1f29f22206bcd2e2477"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.823338 15792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "62407086a4504fef87c0f60140126905"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "103199b0f3a45d9d60fbc34554127414"
server_key_iv: "5c5b3aca2b63d1f29f22206bcd2e2477"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.827208 15792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:45:36.829762 15811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.831013 15792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:36.831125 15792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "62407086a4504fef87c0f60140126905"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "103199b0f3a45d9d60fbc34554127414"
server_key_iv: "5c5b3aca2b63d1f29f22206bcd2e2477"
server_key_version: "encryptionkey@0"
I20260806 18:45:36.831229 15792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:36.835724 15792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:36.836359 15792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:36.836525 15792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:36.837077 15792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:36.838047 15792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:36.838097 15792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.838152 15792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:36.838181 15792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:36.846580 15792 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:45467
I20260806 18:45:36.846657 15966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:45467 every 8 connection(s)
I20260806 18:45:36.847539 15792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:36.853542 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:36.849379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:47195 (local address 127.1.245.126:35285)
0806 18:45:36.849508 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:36.849512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:36.850574 (+ 1062us) server_negotiation.cc:407] Connection header received
0806 18:45:36.851903 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:36.851907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:36.851955 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:36.852029 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:36.853080 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:36.853084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:36.853087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:36.853100 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:36.853120 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:36.853128 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:36.853210 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:36.853346 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 18:45:36.853392 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:45:36.854423 15970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:36.849740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35285 (local address 127.1.245.65:47195)
0806 18:45:36.850425 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:36.850468 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:36.851676 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:36.852212 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:36.852219 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:36.852310 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:36.852798 (+ 488us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:36.852799 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:36.852840 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:36.852908 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:36.853200 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:36.853208 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:36.853214 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:45:36.853377 (+ 163us) client_negotiation.cc:240] Negotiation successful
0806 18:45:36.853622 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260806 18:45:36.854957 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 15792
I20260806 18:45:36.855072 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:45:36.855408 2005 external_mini_cluster.cc:1428] Setting key 3a1bb39ad98e77b74ad1e96f7e385e3e
I20260806 18:45:36.855720 15967 heartbeater.cc:347] Connected to a master server at 127.1.245.126:35285
I20260806 18:45:36.855940 15967 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:36.856338 15967 heartbeater.cc:511] Master 127.1.245.126:35285 requested a full tablet report, sending...
I20260806 18:45:36.857292 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35285
--builtin_ntp_servers=127.1.245.84:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260806 18:45:36.857993 15712 ts_manager.cc:194] Registered new tserver with Master: 62407086a4504fef87c0f60140126905 (127.1.245.65:45467)
W20260806 18:45:36.970052 15972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:36.970345 15972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:36.970403 15972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:36.974421 15972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:36.974634 15972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:36.974733 15972 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:36.979838 15972 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15972
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:36.980985 15972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:36.982447 15972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:36.992111 15981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:36.992095 15978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:36.992096 15983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:36.992400 15972 server_base.cc:1034] running on GCE node
I20260806 18:45:36.992892 15972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:36.993664 15972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:36.994837 15972 hybrid_clock.cc:648] HybridClock initialized: now 1786041936994803 us; error 45 us; skew 500 ppm
I20260806 18:45:36.996814 15972 webserver.cc:460] Webserver started at http://127.1.245.66:32859/ using document root <none> and password file <none>
I20260806 18:45:36.997411 15972 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:36.997480 15972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:36.997696 15972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:36.999845 15972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "fa3fa0bbfec543e3afb417d316b49c8d"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "e416a9a8678b79421c8336528e3a9047"
server_key_iv: "34fbce4c48f6164c577c17abcff868bb"
server_key_version: "encryptionkey@0"
I20260806 18:45:37.000401 15972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "fa3fa0bbfec543e3afb417d316b49c8d"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "e416a9a8678b79421c8336528e3a9047"
server_key_iv: "34fbce4c48f6164c577c17abcff868bb"
server_key_version: "encryptionkey@0"
I20260806 18:45:37.004112 15972 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:45:37.006773 15990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:37.008013 15972 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:37.008127 15972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "fa3fa0bbfec543e3afb417d316b49c8d"
format_stamp: "Formatted at 2026-08-06 18:45:36 on dist-test-slave-fgb3"
server_key: "e416a9a8678b79421c8336528e3a9047"
server_key_iv: "34fbce4c48f6164c577c17abcff868bb"
server_key_version: "encryptionkey@0"
I20260806 18:45:37.008231 15972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:37.012903 15972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:37.013518 15972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:37.013676 15972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:37.014207 15972 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:37.015167 15972 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:37.015218 15972 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:37.015277 15972 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:37.015302 15972 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:37.024101 15972 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:37771
I20260806 18:45:37.024144 16149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:37771 every 8 connection(s)
I20260806 18:45:37.025192 15972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:37.031713 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:37.027302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:38425 (local address 127.1.245.126:35285)
0806 18:45:37.027455 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:37.027459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:37.028619 (+ 1160us) server_negotiation.cc:407] Connection header received
0806 18:45:37.030028 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:37.030031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:37.030080 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:37.030155 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:37.031271 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:37.031275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:37.031277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:37.031295 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:37.031315 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:37.031324 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:37.031404 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:37.031528 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 18:45:37.031573 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:45:37.032591 16154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:37.027786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35285 (local address 127.1.245.66:38425)
0806 18:45:37.028467 (+ 681us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:37.028509 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:37.029816 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:37.030363 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:37.030372 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:37.030467 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:37.030994 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:37.030995 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:37.031040 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:37.031110 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:37.031394 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:37.031403 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:37.031409 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:45:37.031566 (+ 157us) client_negotiation.cc:240] Negotiation successful
0806 18:45:37.031816 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":116,"threads_started":1}
I20260806 18:45:37.033699 16151 heartbeater.cc:347] Connected to a master server at 127.1.245.126:35285
I20260806 18:45:37.033916 16151 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:37.034202 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 15972
I20260806 18:45:37.034379 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:45:37.034379 16151 heartbeater.cc:511] Master 127.1.245.126:35285 requested a full tablet report, sending...
I20260806 18:45:37.034653 2005 external_mini_cluster.cc:1428] Setting key ce3c83824da1536836a91c78a410ba6d
I20260806 18:45:37.035521 15712 ts_manager.cc:194] Registered new tserver with Master: fa3fa0bbfec543e3afb417d316b49c8d (127.1.245.66:37771)
I20260806 18:45:37.036633 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35285
--builtin_ntp_servers=127.1.245.84:43159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 18:45:37.150681 16155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:37.150961 16155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:37.151000 16155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:37.155217 16155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:37.155295 16155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:37.155352 16155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:37.160284 16155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35285
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:37.161459 16155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:37.162978 16155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:37.172897 16169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:37.172868 16162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:37.172876 16164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:37.173171 16155 server_base.cc:1034] running on GCE node
I20260806 18:45:37.173705 16155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:37.174398 16155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:37.175582 16155 hybrid_clock.cc:648] HybridClock initialized: now 1786041937175562 us; error 34 us; skew 500 ppm
I20260806 18:45:37.177557 16155 webserver.cc:460] Webserver started at http://127.1.245.67:40991/ using document root <none> and password file <none>
I20260806 18:45:37.178154 16155 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:37.178227 16155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:37.178475 16155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:37.180553 16155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "a7ffa50afff34790b413050637574207"
format_stamp: "Formatted at 2026-08-06 18:45:37 on dist-test-slave-fgb3"
server_key: "8980dfb5581936798e6b122dfd437eba"
server_key_iv: "1872710ca0f47c51362decf1ad04dd0e"
server_key_version: "encryptionkey@0"
I20260806 18:45:37.181095 16155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "a7ffa50afff34790b413050637574207"
format_stamp: "Formatted at 2026-08-06 18:45:37 on dist-test-slave-fgb3"
server_key: "8980dfb5581936798e6b122dfd437eba"
server_key_iv: "1872710ca0f47c51362decf1ad04dd0e"
server_key_version: "encryptionkey@0"
I20260806 18:45:37.184919 16155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:45:37.187685 16176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:37.189050 16155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:45:37.189175 16155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "a7ffa50afff34790b413050637574207"
format_stamp: "Formatted at 2026-08-06 18:45:37 on dist-test-slave-fgb3"
server_key: "8980dfb5581936798e6b122dfd437eba"
server_key_iv: "1872710ca0f47c51362decf1ad04dd0e"
server_key_version: "encryptionkey@0"
I20260806 18:45:37.189292 16155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:37.194001 16155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:37.194689 16155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:37.194859 16155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:37.195418 16155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:37.196358 16155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:37.196408 16155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:37.196460 16155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:37.196486 16155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:37.204946 16155 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:41995
I20260806 18:45:37.204970 16347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:41995 every 8 connection(s)
I20260806 18:45:37.205899 16155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:37.212479 15764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:37.207883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:35815 (local address 127.1.245.126:35285)
0806 18:45:37.208064 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:37.208068 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:37.209185 (+ 1117us) server_negotiation.cc:407] Connection header received
0806 18:45:37.210598 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:37.210601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:37.210644 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:37.210749 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:37.211936 (+ 1187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:37.211939 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:37.211941 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:45:37.211955 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:37.211977 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:45:37.211986 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:37.212070 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:37.212281 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 18:45:37.212331 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 18:45:37.213369 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 16155
I20260806 18:45:37.213434 16350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:37.208306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35285 (local address 127.1.245.67:35815)
0806 18:45:37.209032 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:37.209076 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:37.210379 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:37.210934 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:37.210941 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:37.211040 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:37.211649 (+ 609us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:45:37.211650 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:37.211695 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:45:37.211775 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:37.212055 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:37.212060 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:37.212063 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 18:45:37.212206 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 18:45:37.212509 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":124,"threads_started":1}
I20260806 18:45:37.213490 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:37.213804 2005 external_mini_cluster.cc:1428] Setting key a3aaf59f72331c53a4413807d7695490
I20260806 18:45:37.214639 16348 heartbeater.cc:347] Connected to a master server at 127.1.245.126:35285
I20260806 18:45:37.214905 16348 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:37.215327 16348 heartbeater.cc:511] Master 127.1.245.126:35285 requested a full tablet report, sending...
I20260806 18:45:37.216341 15711 ts_manager.cc:194] Registered new tserver with Master: a7ffa50afff34790b413050637574207 (127.1.245.67:41995)
I20260806 18:45:37.227615 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 18:45:37.230757 15764 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:45:37.229717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.1.245.126:35285)
0806 18:45:37.229886 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:37.229890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:37.229983 (+ 93us) server_negotiation.cc:407] Connection header received
0806 18:45:37.230136 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:37.230139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:37.230177 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:37.230246 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:37.230626 (+ 380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33596: BlockingRecv error: recv got EOF from 127.0.0.1:33596 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:45:37.230996 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 15792
I20260806 18:45:37.249917 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 15972
I20260806 18:45:37.258622 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 16155
I20260806 18:45:37.267470 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 15666
2026-08-06T18:45:37Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (857 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:45:37 dist-test-slave-fgb3 krb5kdc[16364](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:45:37 dist-test-slave-fgb3 krb5kdc[16364](info): set up 2 sockets
Aug 06 18:45:37 dist-test-slave-fgb3 krb5kdc[16364](info): commencing operation
krb5kdc: starting...
W20260806 18:45:38.394534 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:45:38 dist-test-slave-fgb3 krb5kdc[16364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:45:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:45:38.518447 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35889
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:38.633901 16380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:38.634174 16380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:38.634217 16380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:38.638242 16380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:45:38.638334 16380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:38.638353 16380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:38.638370 16380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:45:38.638404 16380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:45:38.644129 16380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:38.645313 16380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:38.646442 16380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:38.655028 16393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:38.655011 16395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:38.655009 16392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:38.655306 16380 server_base.cc:1034] running on GCE node
I20260806 18:45:38.655983 16380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:38.657074 16380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:38.658252 16380 hybrid_clock.cc:648] HybridClock initialized: now 1786041938658216 us; error 48 us; skew 500 ppm
Aug 06 18:45:38 dist-test-slave-fgb3 krb5kdc[16364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041938, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:38.660991 16380 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:45:38.661881 16380 webserver.cc:460] Webserver started at http://127.1.245.126:46569/ using document root <none> and password file <none>
I20260806 18:45:38.662446 16380 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:38.662515 16380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:38.662693 16380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:38.664672 16380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "5af2f0dbddcc48d293c27a1a53e0bec6"
format_stamp: "Formatted at 2026-08-06 18:45:38 on dist-test-slave-fgb3"
server_key: "7c10e80b890ee07ed7574b8a82b71586"
server_key_iv: "982a083b3307d9f8c5b6f698486ab2c6"
server_key_version: "encryptionkey@0"
I20260806 18:45:38.665199 16380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "5af2f0dbddcc48d293c27a1a53e0bec6"
format_stamp: "Formatted at 2026-08-06 18:45:38 on dist-test-slave-fgb3"
server_key: "7c10e80b890ee07ed7574b8a82b71586"
server_key_iv: "982a083b3307d9f8c5b6f698486ab2c6"
server_key_version: "encryptionkey@0"
I20260806 18:45:38.668854 16380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260806 18:45:38.671325 16406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:38.672554 16380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:38.672660 16380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "5af2f0dbddcc48d293c27a1a53e0bec6"
format_stamp: "Formatted at 2026-08-06 18:45:38 on dist-test-slave-fgb3"
server_key: "7c10e80b890ee07ed7574b8a82b71586"
server_key_iv: "982a083b3307d9f8c5b6f698486ab2c6"
server_key_version: "encryptionkey@0"
I20260806 18:45:38.672748 16380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:38.689575 16380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:38.692788 16380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:38.692952 16380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:38.699913 16380 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:35889
I20260806 18:45:38.700168 16479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:35889 every 8 connection(s)
I20260806 18:45:38.700913 16380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:45:38.703835 16481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:38.704983 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 16380
I20260806 18:45:38.705099 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:45:38.705410 2005 external_mini_cluster.cc:1428] Setting key 563ac221a324ca54fd7d61a0a89d3fac
I20260806 18:45:38.710522 16481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6: Bootstrap starting.
I20260806 18:45:38.713153 16481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:38.714017 16481 log.cc:824] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:45:38 dist-test-slave-fgb3 krb5kdc[16364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:38.715812 16481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6: No bootstrap required, opened a new log
I20260806 18:45:38.719085 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:38.706768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54094 (local address 127.1.245.126:35889)
0806 18:45:38.707227 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:38.707243 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:38.707279 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:45:38.707921 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:38.707938 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:38.708262 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:38.708544 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:38.709440 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:38.710465 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:38.711598 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:38.712062 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:38.712182 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:38.714779 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:38.714785 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:38.714798 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:38.714827 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:38.716409 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:38.716814 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:38.716820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:38.716824 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:38.716936 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:38.717189 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:38.717193 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:38.717195 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:38.717587 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:38.717763 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:38.718061 (+ 298us) server_negotiation.cc:299] Negotiation successful
0806 18:45:38.718361 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":132,"threads_started":1}
I20260806 18:45:38.719162 16481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af2f0dbddcc48d293c27a1a53e0bec6" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35889 } }
I20260806 18:45:38.719425 16481 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:38.719481 16481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af2f0dbddcc48d293c27a1a53e0bec6, State: Initialized, Role: FOLLOWER
I20260806 18:45:38.719861 16481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af2f0dbddcc48d293c27a1a53e0bec6" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35889 } }
I20260806 18:45:38.719981 16481 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:45:38.720077 16481 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:45:38.720181 16481 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:38.721264 16481 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af2f0dbddcc48d293c27a1a53e0bec6" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35889 } }
I20260806 18:45:38.721575 16481 leader_election.cc:302] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5af2f0dbddcc48d293c27a1a53e0bec6; no voters:
I20260806 18:45:38.721832 16481 leader_election.cc:288] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:45:38.721928 16488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:38.722143 16488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [term 1 LEADER]: Becoming Leader. State: Replica: 5af2f0dbddcc48d293c27a1a53e0bec6, State: Running, Role: LEADER
I20260806 18:45:38.722509 16488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af2f0dbddcc48d293c27a1a53e0bec6" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35889 } }
I20260806 18:45:38.722965 16481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:45:38.724015 16491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5af2f0dbddcc48d293c27a1a53e0bec6. Latest consensus state: current_term: 1 leader_uuid: "5af2f0dbddcc48d293c27a1a53e0bec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af2f0dbddcc48d293c27a1a53e0bec6" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35889 } } }
I20260806 18:45:38.724140 16491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:38.724316 16490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5af2f0dbddcc48d293c27a1a53e0bec6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5af2f0dbddcc48d293c27a1a53e0bec6" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35889 } } }
I20260806 18:45:38.724503 16490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:45:38.725170 16506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:45:38.727896 16506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:45:38.732875 16506 catalog_manager.cc:1347] Generated new cluster ID: ea85252d26e64fe4a07d88c1449d0d68
I20260806 18:45:38.732934 16506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:45:38.745471 16506 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:45:38.746833 16506 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:45:38.751893 16506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6: Generated new TSK 0
I20260806 18:45:38.752816 16506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:45:38.784699 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35889
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:38.898819 16524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:38.899080 16524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:38.899124 16524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:38.903076 16524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:38.903129 16524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:38.903191 16524 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:45:38.908352 16524 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:38.909452 16524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:38.910557 16524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:38.920377 16537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:38.920431 16535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:38.920452 16534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:38.920548 16524 server_base.cc:1034] running on GCE node
I20260806 18:45:38.920971 16524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:38.921600 16524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:38.922775 16524 hybrid_clock.cc:648] HybridClock initialized: now 1786041938922748 us; error 37 us; skew 500 ppm
Aug 06 18:45:38 dist-test-slave-fgb3 krb5kdc[16364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041938, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:38.925765 16524 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:45:38.926726 16524 webserver.cc:460] Webserver started at http://127.1.245.65:37869/ using document root <none> and password file <none>
I20260806 18:45:38.927273 16524 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:38.927340 16524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:38.927541 16524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:38.929533 16524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "20063b8fb1df40eeb944d243f6058c03"
format_stamp: "Formatted at 2026-08-06 18:45:38 on dist-test-slave-fgb3"
server_key: "0cfc424b6fff211f8bcc2113c23542e1"
server_key_iv: "148070441daa37c0a09c6a8f5cd6398e"
server_key_version: "encryptionkey@0"
I20260806 18:45:38.930069 16524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "20063b8fb1df40eeb944d243f6058c03"
format_stamp: "Formatted at 2026-08-06 18:45:38 on dist-test-slave-fgb3"
server_key: "0cfc424b6fff211f8bcc2113c23542e1"
server_key_iv: "148070441daa37c0a09c6a8f5cd6398e"
server_key_version: "encryptionkey@0"
I20260806 18:45:38.933753 16524 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:45:38.936154 16551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:38.937330 16524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:45:38.937443 16524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "20063b8fb1df40eeb944d243f6058c03"
format_stamp: "Formatted at 2026-08-06 18:45:38 on dist-test-slave-fgb3"
server_key: "0cfc424b6fff211f8bcc2113c23542e1"
server_key_iv: "148070441daa37c0a09c6a8f5cd6398e"
server_key_version: "encryptionkey@0"
I20260806 18:45:38.937557 16524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:38.955094 16524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:38.958240 16524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:38.958451 16524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:38.959049 16524 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:38.959892 16524 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:38.959940 16524 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:38.959993 16524 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:38.960021 16524 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:38.968919 16524 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:33931
I20260806 18:45:38.968943 16716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:33931 every 8 connection(s)
I20260806 18:45:38.969852 16524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:45:38.971944 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 16524
I20260806 18:45:38.972059 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:45:38.972325 2005 external_mini_cluster.cc:1428] Setting key 26d6686145d50b35a1e60b39e81f68cb
Aug 06 18:45:38 dist-test-slave-fgb3 krb5kdc[16364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041938, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:45:38.984091 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:38.971806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:40667 (local address 127.1.245.126:35889)
0806 18:45:38.971957 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:38.971961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:38.973101 (+ 1140us) server_negotiation.cc:407] Connection header received
0806 18:45:38.974535 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:38.974538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:38.974581 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:38.974647 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:38.976227 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:38.976787 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:38.977939 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:38.978183 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:38.978274 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:38.980980 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:38.980983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:38.980986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:38.981010 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:38.982342 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:38.982826 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:38.982829 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:38.982830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:38.982905 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:38.983151 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:38.983153 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:38.983154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:38.983311 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:38.983379 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:38.983859 (+ 480us) server_negotiation.cc:299] Negotiation successful
0806 18:45:38.983963 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:45:38.985046 16719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:38.972224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35889 (local address 127.1.245.65:40667)
0806 18:45:38.972748 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:38.972796 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:38.974295 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:38.974832 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:38.974842 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:38.975325 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:38.976088 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:38.976106 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:38.976907 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:38.976910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:38.977820 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:38.977830 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:38.978349 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:38.978355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:38.978542 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:38.979087 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:38.979104 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:38.980808 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:38.982457 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:38.982461 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:38.982473 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:38.982755 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:38.982971 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:38.982973 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:38.982974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:38.983096 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:38.983403 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:38.983408 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:38.983655 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 18:45:38.983821 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 18:45:38.984078 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260806 18:45:38.986354 16717 heartbeater.cc:347] Connected to a master server at 127.1.245.126:35889
I20260806 18:45:38.986591 16717 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:38.987077 16717 heartbeater.cc:511] Master 127.1.245.126:35889 requested a full tablet report, sending...
I20260806 18:45:38.988904 16429 ts_manager.cc:194] Registered new tserver with Master: 20063b8fb1df40eeb944d243f6058c03 (127.1.245.65:33931)
I20260806 18:45:38.990581 16429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:40667
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:45:39.000993 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35889
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:39.113302 16723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:39.113581 16723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:39.113626 16723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:39.117708 16723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:39.117779 16723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:39.117842 16723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:45:39.123251 16723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:39.124405 16723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:39.125478 16723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:39.135059 16732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:39.135149 16734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:39.135237 16723 server_base.cc:1034] running on GCE node
W20260806 18:45:39.135304 16736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:39.135778 16723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:39.136443 16723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:39.137607 16723 hybrid_clock.cc:648] HybridClock initialized: now 1786041939137584 us; error 41 us; skew 500 ppm
Aug 06 18:45:39 dist-test-slave-fgb3 krb5kdc[16364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:39.140502 16723 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:45:39.141474 16723 webserver.cc:460] Webserver started at http://127.1.245.66:36797/ using document root <none> and password file <none>
I20260806 18:45:39.142010 16723 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:39.142071 16723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:39.142241 16723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:39.144343 16723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "d906b01155bd45fdb2ebd6a3ae76ea20"
format_stamp: "Formatted at 2026-08-06 18:45:39 on dist-test-slave-fgb3"
server_key: "8231f9c68bc922e74f8eec690ff7954a"
server_key_iv: "cb29bd012921e8d4de906ba761045691"
server_key_version: "encryptionkey@0"
I20260806 18:45:39.144881 16723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "d906b01155bd45fdb2ebd6a3ae76ea20"
format_stamp: "Formatted at 2026-08-06 18:45:39 on dist-test-slave-fgb3"
server_key: "8231f9c68bc922e74f8eec690ff7954a"
server_key_iv: "cb29bd012921e8d4de906ba761045691"
server_key_version: "encryptionkey@0"
I20260806 18:45:39.148783 16723 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:45:39.151335 16743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:39.152627 16723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:45:39.152747 16723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "d906b01155bd45fdb2ebd6a3ae76ea20"
format_stamp: "Formatted at 2026-08-06 18:45:39 on dist-test-slave-fgb3"
server_key: "8231f9c68bc922e74f8eec690ff7954a"
server_key_iv: "cb29bd012921e8d4de906ba761045691"
server_key_version: "encryptionkey@0"
I20260806 18:45:39.152866 16723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:39.182920 16723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:39.186106 16723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:39.186288 16723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:39.186983 16723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:39.187904 16723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:39.187956 16723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:39.188004 16723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:39.188033 16723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:39.196744 16723 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:42677
I20260806 18:45:39.196775 16909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:42677 every 8 connection(s)
I20260806 18:45:39.197667 16723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:45:39.198163 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 16723
I20260806 18:45:39.198278 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:45:39.198591 2005 external_mini_cluster.cc:1428] Setting key a81bd3eca1e308cd65a4c64325ddbf60
Aug 06 18:45:39 dist-test-slave-fgb3 krb5kdc[16364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
I20260806 18:45:39.211936 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.199761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:45101 (local address 127.1.245.126:35889)
0806 18:45:39.199900 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.199905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.200936 (+ 1031us) server_negotiation.cc:407] Connection header received
0806 18:45:39.202351 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.202355 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.202410 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.202487 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:39.203664 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.204192 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.205289 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.205509 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.205599 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.208218 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:39.208222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:39.208225 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:39.208257 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:39.209779 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:39.210362 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:39.210367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:39.210369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:39.210453 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:39.210754 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:39.210757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:39.210758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:39.210952 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:39.211039 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.211625 (+ 586us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.211754 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260806 18:45:39.212932 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.200261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35889 (local address 127.1.245.66:45101)
0806 18:45:39.200791 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:39.200835 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:39.202119 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:39.202667 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:39.202673 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:39.203041 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:39.203526 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.203539 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.204312 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.204314 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.205173 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.205182 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.205649 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.205654 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.205820 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.206395 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:39.206413 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:39.208020 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:39.209898 (+ 1878us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:39.209904 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:39.209919 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:39.210200 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:39.210532 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:39.210535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:39.210536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:39.210685 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:39.211108 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:39.211114 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:39.211383 (+ 269us) client_negotiation.cc:769] Sending connection context
0806 18:45:39.211565 (+ 182us) client_negotiation.cc:240] Negotiation successful
0806 18:45:39.211912 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":142,"threads_started":1}
I20260806 18:45:39.214200 16911 heartbeater.cc:347] Connected to a master server at 127.1.245.126:35889
I20260806 18:45:39.214478 16911 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:39.215040 16911 heartbeater.cc:511] Master 127.1.245.126:35889 requested a full tablet report, sending...
I20260806 18:45:39.216126 16429 ts_manager.cc:194] Registered new tserver with Master: d906b01155bd45fdb2ebd6a3ae76ea20 (127.1.245.66:42677)
I20260806 18:45:39.216704 16429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:45101
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:45:39.227433 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35889
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:45:39.342737 16921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:45:39.343011 16921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:45:39.343061 16921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:45:39.347061 16921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:45:39.347121 16921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:45:39.347182 16921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:45:39.352469 16921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:35889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:45:39.353538 16921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:45:39.354650 16921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:45:39.364104 16930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:39.364147 16934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:45:39.364107 16932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:45:39.364382 16921 server_base.cc:1034] running on GCE node
I20260806 18:45:39.364858 16921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:45:39.365499 16921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:45:39.366644 16921 hybrid_clock.cc:648] HybridClock initialized: now 1786041939366626 us; error 27 us; skew 500 ppm
Aug 06 18:45:39 dist-test-slave-fgb3 krb5kdc[16364](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:45:39.369241 16921 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:45:39.370134 16921 webserver.cc:460] Webserver started at http://127.1.245.67:45839/ using document root <none> and password file <none>
I20260806 18:45:39.370716 16921 fs_manager.cc:359] Metadata directory not provided
I20260806 18:45:39.370786 16921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:45:39.370996 16921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:45:39.373000 16921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82"
format_stamp: "Formatted at 2026-08-06 18:45:39 on dist-test-slave-fgb3"
server_key: "3177fc48c3a67ae8d15b457551b7fb7f"
server_key_iv: "3bf28168be516701a69620c2f0601883"
server_key_version: "encryptionkey@0"
I20260806 18:45:39.373539 16921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82"
format_stamp: "Formatted at 2026-08-06 18:45:39 on dist-test-slave-fgb3"
server_key: "3177fc48c3a67ae8d15b457551b7fb7f"
server_key_iv: "3bf28168be516701a69620c2f0601883"
server_key_version: "encryptionkey@0"
I20260806 18:45:39.377389 16921 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:45:39.379789 16945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:39.381006 16921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:45:39.381112 16921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82"
format_stamp: "Formatted at 2026-08-06 18:45:39 on dist-test-slave-fgb3"
server_key: "3177fc48c3a67ae8d15b457551b7fb7f"
server_key_iv: "3bf28168be516701a69620c2f0601883"
server_key_version: "encryptionkey@0"
I20260806 18:45:39.381217 16921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:45:39.396879 16921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:45:39.399722 16921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:45:39.399878 16921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:45:39.400424 16921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:45:39.401224 16921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:45:39.401270 16921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:39.401310 16921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:45:39.401330 16921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:45:39.409919 16921 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:46155
I20260806 18:45:39.409955 17115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:46155 every 8 connection(s)
I20260806 18:45:39.410811 16921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:45:39.414474 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 16921
I20260806 18:45:39.414585 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:45:39.414849 2005 external_mini_cluster.cc:1428] Setting key 1b5dd662e98c50c2fb716f5f7b9dd155
Aug 06 18:45:39 dist-test-slave-fgb3 krb5kdc[16364](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041939, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:45:39.423698 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.412596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:33055 (local address 127.1.245.126:35889)
0806 18:45:39.412743 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.412747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.413537 (+ 790us) server_negotiation.cc:407] Connection header received
0806 18:45:39.414989 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.414993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.415048 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.415119 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:39.416336 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.416918 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.418011 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.418241 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.418353 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.420590 (+ 2237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:39.420594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:39.420596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:39.420620 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:39.421939 (+ 1319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:39.422382 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:39.422385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:39.422386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:39.422463 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:39.422747 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:39.422749 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:39.422750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:39.422915 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:39.422988 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.423486 (+ 498us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.423582 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:45:39.424569 17120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.412944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35889 (local address 127.1.245.67:33055)
0806 18:45:39.413382 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:39.413424 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:39.414747 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:39.415300 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:39.415306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:39.415720 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:45:39.416202 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.416222 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.417028 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.417031 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.417889 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.417898 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.418383 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.418386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.418537 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.419057 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:45:39.419072 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:45:39.420432 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:45:39.422022 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:39.422026 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:39.422036 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:39.422260 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:39.422533 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:45:39.422535 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:45:39.422537 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:45:39.422660 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:45:39.422988 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:45:39.422992 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:45:39.423246 (+ 254us) client_negotiation.cc:769] Sending connection context
0806 18:45:39.423432 (+ 186us) client_negotiation.cc:240] Negotiation successful
0806 18:45:39.423679 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":105,"threads_started":1}
I20260806 18:45:39.425650 17116 heartbeater.cc:347] Connected to a master server at 127.1.245.126:35889
I20260806 18:45:39.425868 17116 heartbeater.cc:464] Registering TS with master...
I20260806 18:45:39.426345 17116 heartbeater.cc:511] Master 127.1.245.126:35889 requested a full tablet report, sending...
I20260806 18:45:39.427428 16429 ts_manager.cc:194] Registered new tserver with Master: 49165dbd29e24c9c8bbc3f0d28c1bc82 (127.1.245.67:46155)
I20260806 18:45:39.428043 16429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:33055
I20260806 18:45:39.428525 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:45:39.436933 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.430405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54096 (local address 127.1.245.126:35889)
0806 18:45:39.430536 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.430540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.430696 (+ 156us) server_negotiation.cc:407] Connection header received
0806 18:45:39.430812 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.430814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.430849 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.430908 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:45:39.431510 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.431982 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.432761 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.432949 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.433016 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.434357 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:45:39.434361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:45:39.434364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:45:39.434387 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:45:39.435454 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:39.435863 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:39.435866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:39.435867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:39.435949 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:45:39.436215 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:45:39.436217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:45:39.436218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:45:39.436424 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:45:39.436524 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.436695 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.436787 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:45:39.439952 16428 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54096:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:45:39.443032 16428 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:45:39.457544 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.452077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:46155 (local address 127.0.0.1:52758)
0806 18:45:39.452483 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:39.452497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:39.452588 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:39.453292 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:39.453295 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:39.453312 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:39.453465 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.453471 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.454938 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.454941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.456189 (+ 1248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.456223 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.456980 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.456984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.457105 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.457146 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:45:39.457256 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:45:39.457392 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":107,"thread_start_us":94,"threads_started":1}
I20260806 18:45:39.457553 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.452337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.1.245.67:46155)
0806 18:45:39.452833 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.452845 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.452867 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:45:39.452940 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.452946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.453135 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.453445 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:39.453590 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.454799 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.456311 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.456867 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.456965 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.457124 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.457337 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.457432 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":167,"threads_started":1}
I20260806 18:45:39.457763 17134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.451323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:42677 (local address 127.0.0.1:53434)
0806 18:45:39.451804 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:39.451852 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:39.451983 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:39.453439 (+ 1456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:39.453442 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:39.453457 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:39.453549 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.453555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.454971 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.454974 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.455990 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.456000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.457289 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.457293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.457427 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.457451 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:45:39.457534 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 18:45:39.457605 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":4,"thread_start_us":61,"threads_started":1}
I20260806 18:45:39.458096 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.451601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42854 (local address 127.1.245.65:33931)
0806 18:45:39.452236 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.452251 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.452661 (+ 410us) server_negotiation.cc:407] Connection header received
0806 18:45:39.452902 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.452910 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.453115 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.453258 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:39.453650 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.454463 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.456084 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.456989 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.457089 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.457223 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.457831 (+ 608us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.457931 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":522,"thread_start_us":81,"threads_started":1}
I20260806 18:45:39.458106 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.451975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53434 (local address 127.1.245.66:42677)
0806 18:45:39.452833 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.452845 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.452867 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:45:39.452941 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.452946 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.453135 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.453283 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:39.453679 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.454801 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.456126 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.456983 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.457092 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.457223 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.457881 (+ 658us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.457983 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":109,"threads_started":1}
I20260806 18:45:39.457763 17135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.451455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:33931 (local address 127.0.0.1:42854)
0806 18:45:39.452379 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:39.452392 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:39.452737 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:39.453263 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:39.453269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:39.453291 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:39.453559 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.453562 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.454589 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.454591 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.455954 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.455966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.457289 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.457293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.457492 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.457511 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:45:39.457567 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 18:45:39.457620 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":76,"spinlock_wait_cycles":2944,"thread_start_us":127,"threads_started":1}
I20260806 18:45:39.460690 17015 tablet_service.cc:1467] Processing CreateTablet for tablet f7e50f9c169c4734b63ca74040199e2a (DEFAULT_TABLE table=test-table [id=4557a5f4139146e2b640cf16a1c035c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:39.460899 16812 tablet_service.cc:1467] Processing CreateTablet for tablet f7e50f9c169c4734b63ca74040199e2a (DEFAULT_TABLE table=test-table [id=4557a5f4139146e2b640cf16a1c035c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:39.460899 16616 tablet_service.cc:1467] Processing CreateTablet for tablet f7e50f9c169c4734b63ca74040199e2a (DEFAULT_TABLE table=test-table [id=4557a5f4139146e2b640cf16a1c035c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:45:39.461760 17015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e50f9c169c4734b63ca74040199e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:39.461905 16812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e50f9c169c4734b63ca74040199e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:39.461905 16616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7e50f9c169c4734b63ca74040199e2a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:45:39.467710 17141 tablet_bootstrap.cc:492] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Bootstrap starting.
I20260806 18:45:39.469244 17143 tablet_bootstrap.cc:492] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Bootstrap starting.
I20260806 18:45:39.469244 17144 tablet_bootstrap.cc:492] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03: Bootstrap starting.
I20260806 18:45:39.470664 17141 tablet_bootstrap.cc:654] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:39.471568 17144 tablet_bootstrap.cc:654] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:39.471827 17141 log.cc:824] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:39.472033 17143 tablet_bootstrap.cc:654] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Neither blocks nor log segments found. Creating new log.
I20260806 18:45:39.472829 17144 log.cc:824] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:39.472828 17143 log.cc:824] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Log is configured to *not* fsync() on all Append() calls
I20260806 18:45:39.473757 17141 tablet_bootstrap.cc:492] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: No bootstrap required, opened a new log
I20260806 18:45:39.473945 17141 ts_tablet_manager.cc:1397] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:45:39.474831 17144 tablet_bootstrap.cc:492] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03: No bootstrap required, opened a new log
I20260806 18:45:39.474831 17143 tablet_bootstrap.cc:492] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: No bootstrap required, opened a new log
I20260806 18:45:39.475023 17144 ts_tablet_manager.cc:1397] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:45:39.475023 17143 ts_tablet_manager.cc:1397] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:45:39.477430 17141 raft_consensus.cc:348] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.477658 17141 raft_consensus.cc:374] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:39.477730 17141 raft_consensus.cc:729] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49165dbd29e24c9c8bbc3f0d28c1bc82, State: Initialized, Role: FOLLOWER
I20260806 18:45:39.478201 17141 consensus_queue.cc:260] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.478955 17116 heartbeater.cc:503] Master 127.1.245.126:35889 was elected leader, sending a full tablet report...
I20260806 18:45:39.478840 17143 raft_consensus.cc:348] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.479123 17143 raft_consensus.cc:374] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:39.479182 17143 raft_consensus.cc:729] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d906b01155bd45fdb2ebd6a3ae76ea20, State: Initialized, Role: FOLLOWER
I20260806 18:45:39.479239 17141 ts_tablet_manager.cc:1428] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260806 18:45:39.479550 17144 raft_consensus.cc:348] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.479622 17143 consensus_queue.cc:260] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.479758 17144 raft_consensus.cc:374] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:45:39.479838 17144 raft_consensus.cc:729] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20063b8fb1df40eeb944d243f6058c03, State: Initialized, Role: FOLLOWER
I20260806 18:45:39.480262 17144 consensus_queue.cc:260] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.480381 16911 heartbeater.cc:503] Master 127.1.245.126:35889 was elected leader, sending a full tablet report...
I20260806 18:45:39.480612 17143 ts_tablet_manager.cc:1428] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:45:39.481159 16717 heartbeater.cc:503] Master 127.1.245.126:35889 was elected leader, sending a full tablet report...
I20260806 18:45:39.481232 17144 ts_tablet_manager.cc:1428] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 18:45:39.661795 17117 tablet.cc:2369] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:45:39.698953 16912 tablet.cc:2369] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:45:39.701211 17154 raft_consensus.cc:482] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:45:39.701393 17154 raft_consensus.cc:504] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.702603 17154 leader_election.cc:288] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20063b8fb1df40eeb944d243f6058c03 (127.1.245.65:33931), 49165dbd29e24c9c8bbc3f0d28c1bc82 (127.1.245.67:46155)
I20260806 18:45:39.707008 17136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.703077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:45023 (local address 127.1.245.65:33931)
0806 18:45:39.703206 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.703211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.703657 (+ 446us) server_negotiation.cc:407] Connection header received
0806 18:45:39.703759 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.703762 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.703813 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.703883 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:39.704166 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.704772 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.705817 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.706420 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.706545 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.706601 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.706793 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.706876 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:45:39.707121 16914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.702901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:46155 (local address 127.1.245.66:42579)
0806 18:45:39.703113 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:39.703129 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:39.703237 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:39.703523 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:39.703526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:39.703551 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:39.703669 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.703675 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.704631 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.704634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.705836 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.705847 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.706700 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.706703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.706819 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.706836 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 18:45:39.706902 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:45:39.706989 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260806 18:45:39.707120 17140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.703014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:42579 (local address 127.1.245.67:46155)
0806 18:45:39.703141 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.703145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.703188 (+ 43us) server_negotiation.cc:407] Connection header received
0806 18:45:39.703356 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.703358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.703413 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.703489 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:45:39.703798 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.704508 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.705972 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.706597 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.706704 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.706750 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.706936 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.707004 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:45:39.707410 17157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.702903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:33931 (local address 127.1.245.66:45023)
0806 18:45:39.703554 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:39.703567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:45:39.703670 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:45:39.703913 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:45:39.703916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:45:39.703936 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:45:39.704051 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.704057 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.704881 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.704884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.705685 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:45:39.705695 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.706533 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.706536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:45:39.706636 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.706662 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:45:39.707195 (+ 533us) client_negotiation.cc:240] Negotiation successful
0806 18:45:39.707274 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"mutex_wait_us":318,"thread_start_us":102,"threads_started":1}
I20260806 18:45:39.707863 17049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e50f9c169c4734b63ca74040199e2a" candidate_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" is_pre_election: true
I20260806 18:45:39.708099 16647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e50f9c169c4734b63ca74040199e2a" candidate_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20063b8fb1df40eeb944d243f6058c03" is_pre_election: true
I20260806 18:45:39.708226 17049 raft_consensus.cc:2454] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d906b01155bd45fdb2ebd6a3ae76ea20 in term 0.
I20260806 18:45:39.708370 16647 raft_consensus.cc:2454] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d906b01155bd45fdb2ebd6a3ae76ea20 in term 0.
I20260806 18:45:39.708755 16747 leader_election.cc:302] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 20063b8fb1df40eeb944d243f6058c03, 49165dbd29e24c9c8bbc3f0d28c1bc82, d906b01155bd45fdb2ebd6a3ae76ea20; no voters:
I20260806 18:45:39.709057 17154 raft_consensus.cc:2781] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:45:39.709141 17154 raft_consensus.cc:482] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:45:39.709208 17154 raft_consensus.cc:3037] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:39.710254 17154 raft_consensus.cc:504] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.710645 17154 leader_election.cc:288] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [CANDIDATE]: Term 1 election: Requested vote from peers 20063b8fb1df40eeb944d243f6058c03 (127.1.245.65:33931), 49165dbd29e24c9c8bbc3f0d28c1bc82 (127.1.245.67:46155)
I20260806 18:45:39.711004 16647 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e50f9c169c4734b63ca74040199e2a" candidate_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20063b8fb1df40eeb944d243f6058c03"
I20260806 18:45:39.711055 17049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7e50f9c169c4734b63ca74040199e2a" candidate_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82"
I20260806 18:45:39.711138 16647 raft_consensus.cc:3037] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:39.711158 17049 raft_consensus.cc:3037] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:45:39.712384 17049 raft_consensus.cc:2454] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d906b01155bd45fdb2ebd6a3ae76ea20 in term 1.
I20260806 18:45:39.712384 16647 raft_consensus.cc:2454] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d906b01155bd45fdb2ebd6a3ae76ea20 in term 1.
I20260806 18:45:39.712670 16748 leader_election.cc:302] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20063b8fb1df40eeb944d243f6058c03, d906b01155bd45fdb2ebd6a3ae76ea20; no voters:
I20260806 18:45:39.712836 17154 raft_consensus.cc:2781] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:45:39.713090 17154 raft_consensus.cc:686] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 1 LEADER]: Becoming Leader. State: Replica: d906b01155bd45fdb2ebd6a3ae76ea20, State: Running, Role: LEADER
I20260806 18:45:39.713438 17154 consensus_queue.cc:237] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } }
I20260806 18:45:39.716562 16429 catalog_manager.cc:5495] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 reported cstate change: term changed from 0 to 1, leader changed from <none> to d906b01155bd45fdb2ebd6a3ae76ea20 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d906b01155bd45fdb2ebd6a3ae76ea20" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 42677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 } health_report { overall_health: UNKNOWN } } }
W20260806 18:45:39.721246 16718 tablet.cc:2369] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:45:39.751641 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.748309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.1.245.66:42677)
0806 18:45:39.748419 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0806 18:45:39.748424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:45:39.748436 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:45:39.748558 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:45:39.748560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:45:39.748619 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:45:39.748710 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:45:39.748909 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.749409 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.750280 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:45:39.750501 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:45:39.750572 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:45:39.750871 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:45:39.751295 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:45:39.751383 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:45:39.751455 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 18:45:39.751508 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:45:39.757504 16647 raft_consensus.cc:1261] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 1 FOLLOWER]: Refusing update from remote peer d906b01155bd45fdb2ebd6a3ae76ea20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:39.757506 17049 raft_consensus.cc:1261] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 1 FOLLOWER]: Refusing update from remote peer d906b01155bd45fdb2ebd6a3ae76ea20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:45:39.758195 17159 consensus_queue.cc:1035] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20063b8fb1df40eeb944d243f6058c03" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 33931 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:39.758392 17154 consensus_queue.cc:1035] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49165dbd29e24c9c8bbc3f0d28c1bc82" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 46155 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:45:39.765232 17168 mvcc.cc:205] Tried to move back new op lower bound from 7315627785239453696 to 7315627785068843008. Current Snapshot: MvccSnapshot[applied={T|T < 7315627785239453696}]
I20260806 18:45:39.766126 17171 mvcc.cc:205] Tried to move back new op lower bound from 7315627785239453696 to 7315627785068843008. Current Snapshot: MvccSnapshot[applied={T|T < 7315627785239453696}]
I20260806 18:45:39.768355 17175 mvcc.cc:205] Tried to move back new op lower bound from 7315627785239453696 to 7315627785068843008. Current Snapshot: MvccSnapshot[applied={T|T < 7315627785239453696}]
I20260806 18:45:39.773547 16429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:39.773685 16429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54096:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:45:39.776108 16429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5af2f0dbddcc48d293c27a1a53e0bec6: Sending DeleteTablet for 3 replicas of tablet f7e50f9c169c4734b63ca74040199e2a
I20260806 18:45:39.776888 16616 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e50f9c169c4734b63ca74040199e2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:39 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:42854
I20260806 18:45:39.776898 16812 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e50f9c169c4734b63ca74040199e2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:39 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:53434
I20260806 18:45:39.776988 17015 tablet_service.cc:1514] Processing DeleteTablet for tablet f7e50f9c169c4734b63ca74040199e2a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:45:39 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:52758
I20260806 18:45:39.777431 17190 tablet_replica.cc:330] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: stopping tablet replica
I20260806 18:45:39.777549 17188 tablet_replica.cc:330] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03: stopping tablet replica
I20260806 18:45:39.777825 17188 raft_consensus.cc:2229] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:39.777827 17190 raft_consensus.cc:2229] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:45:39.778146 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 16524
I20260806 18:45:39.778164 17188 raft_consensus.cc:2258] T f7e50f9c169c4734b63ca74040199e2a P 20063b8fb1df40eeb944d243f6058c03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:39.778820 17186 tablet_replica.cc:330] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: stopping tablet replica
I20260806 18:45:39.779099 17186 raft_consensus.cc:2229] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:45:39.779368 17186 raft_consensus.cc:2258] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:39.778240 17190 raft_consensus.cc:2258] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:45:39.782208 17186 ts_tablet_manager.cc:1905] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:39.782208 17190 ts_tablet_manager.cc:1905] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:45:39.784992 17186 ts_tablet_manager.cc:1918] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:39.785127 17186 log.cc:1196] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/f7e50f9c169c4734b63ca74040199e2a
I20260806 18:45:39.785146 17190 ts_tablet_manager.cc:1918] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:45:39.785251 17190 log.cc:1196] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/f7e50f9c169c4734b63ca74040199e2a
I20260806 18:45:39.785529 17186 ts_tablet_manager.cc:1939] T f7e50f9c169c4734b63ca74040199e2a P 49165dbd29e24c9c8bbc3f0d28c1bc82: Deleting consensus metadata
I20260806 18:45:39.785610 17190 ts_tablet_manager.cc:1939] T f7e50f9c169c4734b63ca74040199e2a P d906b01155bd45fdb2ebd6a3ae76ea20: Deleting consensus metadata
I20260806 18:45:39.786835 16408 catalog_manager.cc:4841] TS 49165dbd29e24c9c8bbc3f0d28c1bc82 (127.1.245.67:46155): tablet f7e50f9c169c4734b63ca74040199e2a (table test-table [id=4557a5f4139146e2b640cf16a1c035c1]) successfully deleted
I20260806 18:45:39.787197 16412 catalog_manager.cc:4841] TS d906b01155bd45fdb2ebd6a3ae76ea20 (127.1.245.66:42677): tablet f7e50f9c169c4734b63ca74040199e2a (table test-table [id=4557a5f4139146e2b640cf16a1c035c1]) successfully deleted
W20260806 18:45:39.792984 16409 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:33931 (error 108)
I20260806 18:45:39.793111 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 16723
I20260806 18:45:39.801949 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:45:39.801572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:33931 (local address 127.0.0.1:42860)
0806 18:45:39.801789 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0806 18:45:39.801860 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:33931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260806 18:45:39.802214 16409 catalog_manager.cc:4584] TS 20063b8fb1df40eeb944d243f6058c03 (127.1.245.65:33931): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7e50f9c169c4734b63ca74040199e2a: Network error: Client connection negotiation failed: client connection to 127.1.245.65:33931: connect: Connection refused (error 111)
I20260806 18:45:39.807276 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 16921
I20260806 18:45:39.828154 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 16380
2026-08-06T18:45:39Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2568 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:45:39 dist-test-slave-fgb3 krb5kdc[17196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:45:39 dist-test-slave-fgb3 krb5kdc[17196](info): set up 2 sockets
Aug 06 18:45:39 dist-test-slave-fgb3 krb5kdc[17196](info): commencing operation
krb5kdc: starting...
W20260806 18:45:40.965698 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:45:41 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:45:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:45:41Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.245.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.245.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/rangeradmin_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/rangeradmin_127.1.245.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.245.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.245.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/rangerlookup_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/rangerlookup_127.1.245.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.245.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.245.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/HTTP_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/HTTP_127.1.245.84@KRBTEST.COM.keytab.
I20260806 18:45:41.135268 2005 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskesmHnt/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/postgres -l logfile start
2026-08-06 18:45:43.571 UTC [17241] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:45:43.571 UTC [17241] LOG: listening on IPv4 address "127.1.245.84", port 46155
2026-08-06 18:45:43.575 UTC [17241] LOG: listening on Unix socket "/tmp/.s.PGSQL.46155"
2026-08-06 18:45:43.595 UTC [17246] LOG: database system was shut down at 2026-08-06 18:45:42 UTC
2026-08-06 18:45:43.599 UTC [17241] LOG: database system is ready to accept connections
I20260806 18:45:44.637826 2005 mini_postgres.cc:96] Postgres bound to 46155
2026-08-06 18:45:44.644 UTC [17255] FATAL: database "slave" does not exist
127.1.245.84:46155 - accepting connections
I20260806 18:45:44.645239 2005 mini_ranger.cc:161] Starting Ranger...
I20260806 18:45:44.660091 2005 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:45:45.118067 2005 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:45:45.118165 2005 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-admin
2026-08-06 18:45:45,411 [I] DB FLAVOR :POSTGRES
2026-08-06 18:45:45,412 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:45:45,412 [I] Checking connection..
2026-08-06 18:45:45,412 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:45:45,667 [I] Checking connection passed.
2026-08-06 18:45:45,667 [I] --------- Verifying version history table ---------
2026-08-06 18:45:45,667 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:45:45,927 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:45:45,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:45:46,169 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:45:46,169 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:45:46,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:45:46.402 UTC [17361] WARNING: there is no transaction in progress
2026-08-06 18:45:46,410 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:45:46,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:45:46,653 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:45:46,653 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:45:46,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:45:46,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:45:47,169 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fgb3.c.gcp-upstream.internal','N') ;"
2026-08-06 18:45:47,417 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:45:47,690 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:45:47,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:45:47,952 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:45:47,953 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:45:48,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:45:48,463 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:45:48,464 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:45:49.430 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.447 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.463 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.478 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.494 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.512 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.577 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.584 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.599 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.610 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.620 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.634 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.644 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.650 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.657 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:49.664 UTC [17514] WARNING: there is no transaction in progress
2026-08-06 18:45:50,306 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:45:50,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:45:50,539 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:45:50,539 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:45:50,794 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:45:50,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:45:51,051 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:45:51,051 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:45:51,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:45:51,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fgb3.c.gcp-upstream.internal';"
2026-08-06 18:45:51,762 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:45:51,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:45:51,994 [I] DB_PATCHES have already been applied
I20260806 18:45:52.000128 2005 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-admin:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskesmHnt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:45:52.000195 2005 mini_ranger.cc:193] Using host: 127.1.245.84
I20260806 18:45:52.004132 2005 mini_ranger.cc:227] Ranger admin URL: http://127.1.245.84:38635
Aug 06, 2026 6:45:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:45:52 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:45:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:45:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:45:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:45:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:45:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:45:53.182626 2005 mini_ranger.cc:160] Time spent starting Ranger: real 8.537s user 0.001s sys 0.010s
Aug 06, 2026 6:46:13 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:46:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 6:46:13 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:46:13 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:46:14 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:46:15.565582 2005 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.245.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.245.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/rangerkms_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/rangerkms_127.1.245.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.245.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.245.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.245.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/HTTP_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.245.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/HTTP_127.1.245.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:46:15 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041975, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:46:15.665910 2005 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:46:15.678727 2005 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:46:16.567792 2005 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:46:16.567883 2005 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms
I20260806 18:46:16.567979 2005 mini_ranger_kms.cc:227] Using postgres at 127.1.245.84:46155
2026-08-06 18:46:17,224 [I] DB FLAVOR :POSTGRES
2026-08-06 18:46:17,224 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:46:17,224 [I] Checking connection
2026-08-06 18:46:17,224 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:46:17,583 [I] connection success
2026-08-06 18:46:17,583 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:46:17,583 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:46:17,583 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:46:17,842 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:46:17,843 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:46:17,843 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:46:17,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:46155/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:46:18,141 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:46:18.592176 2005 mini_ranger_kms.cc:310] Created kms service
I20260806 18:46:18.794973 2005 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:46:18.841238 2005 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:46:18 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041975, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:46:19.263077 2005 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:46:19.263406 2005 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms:/tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:46:19.263444 2005 mini_ranger_kms.cc:240] Using host: 127.1.245.84
I20260806 18:46:19.268890 2005 mini_ranger_kms.cc:276] Ranger KMS PID: 17786
I20260806 18:46:19.269042 2005 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.245.84:57871
Aug 06, 2026 6:46:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:46:20 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:46:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:46:20 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:46:21 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:46:21 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260806 18:46:21.748137 2005 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.082s user 0.027s sys 0.595s
I20260806 18:46:21.748291 2005 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:46:21.748431 2005 mini_ranger_kms.cc:401] 127.1.245.84:57871/kms/v1/keys
Aug 06 18:46:21 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041981, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.245.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 18:46:22 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041981, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.245.84@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
Aug 06 18:46:22 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041981, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.245.84@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
Aug 06 18:46:25 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:46:25.357834 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:40449
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:40449
--ranger_config_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:57871/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:46:25.514156 17860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:46:25.514541 17860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:46:25.514693 17860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:46:25.518976 17860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:46:25.519061 17860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:46:25.519094 17860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:46:25.519127 17860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:46:25.519162 17860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:46:25.525297 17860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.245.84:57871/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:40449
--ranger_config_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:40449
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:46:25.526551 17860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:46:25.527688 17860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:46:25.537612 17872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:46:25.537789 17860 server_base.cc:1034] running on GCE node
W20260806 18:46:25.537875 17868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:46:25.537976 17869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:46:25.538754 17860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:46:25.539979 17860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:46:25.541144 17860 hybrid_clock.cc:648] HybridClock initialized: now 1786041985541086 us; error 80 us; skew 500 ppm
Aug 06 18:46:25 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041985, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:46:25.547863 17860 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:46:25.548931 17860 webserver.cc:460] Webserver started at http://127.1.245.126:40215/ using document root <none> and password file <none>
I20260806 18:46:25.549494 17860 fs_manager.cc:359] Metadata directory not provided
I20260806 18:46:25.549556 17860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:46:25.549763 17860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:46:25 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041985, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:46:25.619673 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "363f9cec88874f7aac0f93b5225f06f0"
format_stamp: "Formatted at 2026-08-06 18:46:25 on dist-test-slave-fgb3"
server_key: "b8090e7553826f7ccd528987b8381161"
server_key_iv: "49ac38d13550ba7a3d09f80d33091944"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:25.620371 17860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "363f9cec88874f7aac0f93b5225f06f0"
format_stamp: "Formatted at 2026-08-06 18:46:25 on dist-test-slave-fgb3"
server_key: "b8090e7553826f7ccd528987b8381161"
server_key_iv: "49ac38d13550ba7a3d09f80d33091944"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:25.624500 17860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:46:25.641578 17885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:25.643181 17860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:46:25.643326 17860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "363f9cec88874f7aac0f93b5225f06f0"
format_stamp: "Formatted at 2026-08-06 18:46:25 on dist-test-slave-fgb3"
server_key: "b8090e7553826f7ccd528987b8381161"
server_key_iv: "49ac38d13550ba7a3d09f80d33091944"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:25.643467 17860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:46:25.663035 17860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:46:25.666553 17860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:46:25.666853 17860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:46:25.683859 17860 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:40449
I20260806 18:46:25.683873 17964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:40449 every 8 connection(s)
I20260806 18:46:25.685420 17860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:46:25.687444 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 17860
I20260806 18:46:25.687561 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:46:25.687903 2005 external_mini_cluster.cc:1428] Setting key 9223245f79a84556e778a3ad92123b4b
I20260806 18:46:25.690654 17966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:46:25.698374 17966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0: Bootstrap starting.
Aug 06 18:46:25 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041985, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:46:25.701839 17966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0: Neither blocks nor log segments found. Creating new log.
I20260806 18:46:25.703356 17966 log.cc:824] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:46:25.705679 17966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0: No bootstrap required, opened a new log
I20260806 18:46:25.708325 17969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:25.689955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.1.245.126:40449)
0806 18:46:25.690573 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:25.690626 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:25.690678 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:46:25.692102 (+ 1424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:25.692129 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:25.692574 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:25.692973 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:46:25.693882 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:25.695867 (+ 1985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:25.697105 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:25.697884 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:25.698068 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:25.701390 (+ 3322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:46:25.701401 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:46:25.701422 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:46:25.701462 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:46:25.704444 (+ 2982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:25.704961 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:25.704969 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:25.704975 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:25.705121 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:25.705439 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:25.705442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:25.705444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:25.705998 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:46:25.706201 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:25.706727 (+ 526us) server_negotiation.cc:299] Negotiation successful
0806 18:46:25.707215 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":146,"threads_started":1}
I20260806 18:46:25.710284 17966 raft_consensus.cc:348] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363f9cec88874f7aac0f93b5225f06f0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40449 } }
I20260806 18:46:25.710614 17966 raft_consensus.cc:374] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:46:25.710704 17966 raft_consensus.cc:729] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 363f9cec88874f7aac0f93b5225f06f0, State: Initialized, Role: FOLLOWER
I20260806 18:46:25.711197 17966 consensus_queue.cc:260] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363f9cec88874f7aac0f93b5225f06f0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40449 } }
I20260806 18:46:25.711478 17966 raft_consensus.cc:388] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:46:25.711548 17966 raft_consensus.cc:482] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:46:25.711632 17966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:46:25.712821 17966 raft_consensus.cc:504] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363f9cec88874f7aac0f93b5225f06f0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40449 } }
I20260806 18:46:25.713148 17966 leader_election.cc:302] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 363f9cec88874f7aac0f93b5225f06f0; no voters:
I20260806 18:46:25.713470 17966 leader_election.cc:288] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:46:25.713691 17972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:46:25.714093 17972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [term 1 LEADER]: Becoming Leader. State: Replica: 363f9cec88874f7aac0f93b5225f06f0, State: Running, Role: LEADER
I20260806 18:46:25.714433 17972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363f9cec88874f7aac0f93b5225f06f0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40449 } }
I20260806 18:46:25.714922 17966 sys_catalog.cc:564] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:46:25.715716 17972 sys_catalog.cc:455] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 363f9cec88874f7aac0f93b5225f06f0. Latest consensus state: current_term: 1 leader_uuid: "363f9cec88874f7aac0f93b5225f06f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363f9cec88874f7aac0f93b5225f06f0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40449 } } }
I20260806 18:46:25.715935 17972 sys_catalog.cc:458] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:46:25.716423 17973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "363f9cec88874f7aac0f93b5225f06f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "363f9cec88874f7aac0f93b5225f06f0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40449 } } }
I20260806 18:46:25.716653 17973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:46:25.716437 17981 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:46:25.721736 17981 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:46:25.724781 17966 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:46:25.736743 17981 catalog_manager.cc:1347] Generated new cluster ID: 533f1e68555840808608c95c2be0e095
I20260806 18:46:25.736830 17981 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:46:25.747206 17981 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:46:25.748610 17981 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:46:25.770130 17981 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0: Generated new TSK 0
I20260806 18:46:25.771124 17981 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:46:26.727526 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:57871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40449
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:46:26.931891 18035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:46:26.932292 18035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:46:26.932372 18035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:46:26.938438 18035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:46:26.938519 18035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:46:26.938606 18035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:46:26.946803 18035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.245.84:57871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:46:26.948326 18035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:46:26.949592 18035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:46:26.964591 18051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:46:26.964630 18049 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:46:26.965040 18053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:46:26.965559 18035 server_base.cc:1034] running on GCE node
I20260806 18:46:26.966087 18035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:46:26.967012 18035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:46:26.968226 18035 hybrid_clock.cc:648] HybridClock initialized: now 1786041986968209 us; error 37 us; skew 500 ppm
Aug 06 18:46:26 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041986, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:46:26.972638 18035 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:46:26.973879 18035 webserver.cc:460] Webserver started at http://127.1.245.65:43291/ using document root <none> and password file <none>
I20260806 18:46:26.974697 18035 fs_manager.cc:359] Metadata directory not provided
I20260806 18:46:26.974807 18035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:46:26.975152 18035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:46:26 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041986, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:46:26.993232 18035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "f5784053dd6348d1932ea2807ec59843"
format_stamp: "Formatted at 2026-08-06 18:46:26 on dist-test-slave-fgb3"
server_key: "24f83940696e5dd7aab0ea3f40bfc32f"
server_key_iv: "b528d2804f22d677c9c7169cc591ac03"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:26.994166 18035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "f5784053dd6348d1932ea2807ec59843"
format_stamp: "Formatted at 2026-08-06 18:46:26 on dist-test-slave-fgb3"
server_key: "24f83940696e5dd7aab0ea3f40bfc32f"
server_key_iv: "b528d2804f22d677c9c7169cc591ac03"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:27.001504 18035 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.001s
I20260806 18:46:27.022608 18063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.024523 18035 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 18:46:27.024688 18035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "f5784053dd6348d1932ea2807ec59843"
format_stamp: "Formatted at 2026-08-06 18:46:26 on dist-test-slave-fgb3"
server_key: "24f83940696e5dd7aab0ea3f40bfc32f"
server_key_iv: "b528d2804f22d677c9c7169cc591ac03"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:27.024852 18035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:46:27.043991 18035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:46:27.047719 18035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:46:27.047991 18035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:46:27.048702 18035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:46:27.049645 18035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:46:27.049705 18035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.049783 18035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:46:27.049814 18035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.070283 18035 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:43471
I20260806 18:46:27.073550 18035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:46:27.074656 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 18035
I20260806 18:46:27.074776 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:46:27.075230 2005 external_mini_cluster.cc:1428] Setting key 0ed2136a434477fd809ac0156a95e905
I20260806 18:46:27.085103 18243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:43471 every 8 connection(s)
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041986, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:46:27.107379 18249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.078463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:49309 (local address 127.1.245.126:40449)
0806 18:46:27.094274 (+ 15811us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:27.094280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:27.094299 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:46:27.094375 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:27.094379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:27.094441 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:27.094552 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:46:27.096247 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.097130 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.099003 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.099389 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.100196 (+ 807us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.102588 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:46:27.102592 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:46:27.102595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:46:27.102624 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:46:27.104566 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:27.105250 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:27.105254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:27.105255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:27.105356 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:27.105804 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:27.105808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:27.105810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:27.106052 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:46:27.106154 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:27.106963 (+ 809us) server_negotiation.cc:299] Negotiation successful
0806 18:46:27.107115 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15704,"thread_start_us":102,"threads_started":1}
I20260806 18:46:27.108129 18248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.077266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40449 (local address 127.1.245.65:49309)
0806 18:46:27.083896 (+ 6630us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:27.083966 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:27.087685 (+ 3719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:27.094797 (+ 7112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:46:27.094808 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:46:27.095370 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:46:27.096063 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.096085 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.097299 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.097303 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.098839 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.098851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.099579 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.099584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.099827 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.100573 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:46:27.100595 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:46:27.102418 (+ 1823us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:46:27.104701 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:46:27.104707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:46:27.104727 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:46:27.105125 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:46:27.105470 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:46:27.105474 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:46:27.105476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:46:27.105673 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:46:27.106177 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:46:27.106184 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:46:27.106564 (+ 380us) client_negotiation.cc:769] Sending connection context
0806 18:46:27.106784 (+ 220us) client_negotiation.cc:240] Negotiation successful
0806 18:46:27.107154 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6359,"thread_start_us":169,"threads_started":1}
I20260806 18:46:27.109547 18246 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40449
I20260806 18:46:27.109835 18246 heartbeater.cc:464] Registering TS with master...
I20260806 18:46:27.110476 18246 heartbeater.cc:511] Master 127.1.245.126:40449 requested a full tablet report, sending...
I20260806 18:46:27.112187 17916 ts_manager.cc:194] Registered new tserver with Master: f5784053dd6348d1932ea2807ec59843 (127.1.245.65:43471)
I20260806 18:46:27.113557 17916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:49309
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:46:27.115531 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:57871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40449
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786041987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:46:27.304250 18255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:46:27.304582 18255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:46:27.304646 18255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:46:27.309347 18255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:46:27.309553 18255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:46:27.309689 18255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:46:27.317680 18255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.245.84:57871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:46:27.319185 18255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:46:27.320261 18255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:46:27.330813 18267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:46:27.330734 18269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:46:27.331208 18255 server_base.cc:1034] running on GCE node
W20260806 18:46:27.331290 18265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:46:27.332043 18255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:46:27.332686 18255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:46:27.333865 18255 hybrid_clock.cc:648] HybridClock initialized: now 1786041987333844 us; error 36 us; skew 500 ppm
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:46:27.337157 18255 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:46:27.338455 18255 webserver.cc:460] Webserver started at http://127.1.245.66:42099/ using document root <none> and password file <none>
I20260806 18:46:27.339219 18255 fs_manager.cc:359] Metadata directory not provided
I20260806 18:46:27.339320 18255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:46:27.339594 18255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:46:27.357587 18255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "f70e36897dcd4537976365f25d7026b9"
format_stamp: "Formatted at 2026-08-06 18:46:27 on dist-test-slave-fgb3"
server_key: "cfb74619675d9535adc3e2c24991108d"
server_key_iv: "a10d99c70a215f13fad5f9518ef496a6"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:27.358681 18255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "f70e36897dcd4537976365f25d7026b9"
format_stamp: "Formatted at 2026-08-06 18:46:27 on dist-test-slave-fgb3"
server_key: "cfb74619675d9535adc3e2c24991108d"
server_key_iv: "a10d99c70a215f13fad5f9518ef496a6"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:27.363162 18255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 18:46:27.381199 18281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.383212 18255 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 18:46:27.383386 18255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "f70e36897dcd4537976365f25d7026b9"
format_stamp: "Formatted at 2026-08-06 18:46:27 on dist-test-slave-fgb3"
server_key: "cfb74619675d9535adc3e2c24991108d"
server_key_iv: "a10d99c70a215f13fad5f9518ef496a6"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:27.383558 18255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:46:27.405882 18255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:46:27.409974 18255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:46:27.410210 18255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:46:27.411108 18255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:46:27.412389 18255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:46:27.412456 18255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.412513 18255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:46:27.412544 18255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.437206 18255 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:36895
I20260806 18:46:27.437270 18448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:36895 every 8 connection(s)
I20260806 18:46:27.438264 18255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:46:27.440780 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 18255
I20260806 18:46:27.440899 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:46:27.441205 2005 external_mini_cluster.cc:1428] Setting key e59d6c334d77bf1f87e9c8e863bb3aa7
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:46:27.455396 18249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.440460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:52923 (local address 127.1.245.126:40449)
0806 18:46:27.440597 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:27.440601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:27.441731 (+ 1130us) server_negotiation.cc:407] Connection header received
0806 18:46:27.443052 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:27.443056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:27.443109 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:27.443179 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:46:27.444401 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.445166 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.447153 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.447556 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.447648 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.451001 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:46:27.451005 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:46:27.451008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:46:27.451040 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:46:27.452864 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:27.453549 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:27.453553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:27.453554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:27.453635 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:27.454001 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:27.454004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:27.454006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:27.454200 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:46:27.454299 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:27.455048 (+ 749us) server_negotiation.cc:299] Negotiation successful
0806 18:46:27.455200 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:46:27.457479 18455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.440908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40449 (local address 127.1.245.66:52923)
0806 18:46:27.441514 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:27.441571 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:27.442819 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:27.443388 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:46:27.443396 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:46:27.443716 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:46:27.444260 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.444275 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.445332 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.445337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.446993 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.447006 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.447899 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.447906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.448166 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.448922 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:46:27.448944 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:46:27.450796 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:46:27.453030 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:46:27.453036 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:46:27.453056 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:46:27.453430 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:46:27.453734 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:46:27.453737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:46:27.453739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:46:27.453894 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:46:27.454338 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:46:27.454345 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:46:27.454717 (+ 372us) client_negotiation.cc:769] Sending connection context
0806 18:46:27.455922 (+ 1205us) client_negotiation.cc:240] Negotiation successful
0806 18:46:27.456317 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":176,"threads_started":1}
I20260806 18:46:27.459501 18450 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40449
I20260806 18:46:27.459770 18450 heartbeater.cc:464] Registering TS with master...
I20260806 18:46:27.460378 18450 heartbeater.cc:511] Master 127.1.245.126:40449 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:46:27.461709 17916 ts_manager.cc:194] Registered new tserver with Master: f70e36897dcd4537976365f25d7026b9 (127.1.245.66:36895)
I20260806 18:46:27.462569 17916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:52923
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:46:27.486253 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:57871/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40449
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44211
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:46:27.677292 18461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:46:27.677709 18461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:46:27.677793 18461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:46:27.684067 18461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:46:27.684149 18461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:46:27.684239 18461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:46:27.692539 18461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44211
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.245.84:57871/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40449
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:46:27.694164 18461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:46:27.695734 18461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:46:27.712352 18461 server_base.cc:1034] running on GCE node
W20260806 18:46:27.712616 18473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:46:27.712973 18469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:46:27.714884 18470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:46:27.715533 18461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:46:27.716399 18461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:46:27.717583 18461 hybrid_clock.cc:648] HybridClock initialized: now 1786041987717513 us; error 86 us; skew 500 ppm
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:46:27.722535 18461 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:46:27.723728 18461 webserver.cc:460] Webserver started at http://127.1.245.67:38351/ using document root <none> and password file <none>
I20260806 18:46:27.724466 18461 fs_manager.cc:359] Metadata directory not provided
I20260806 18:46:27.724545 18461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:46:27.724795 18461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:46:27.744571 18461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "332afd749ddf4a0bac2d547923cb80e9"
format_stamp: "Formatted at 2026-08-06 18:46:27 on dist-test-slave-fgb3"
server_key: "702ddf623cb591e5f2160a9adb340ff6"
server_key_iv: "d0027ec09f83d865e8a5df038b42a33d"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:27.745636 18461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "332afd749ddf4a0bac2d547923cb80e9"
format_stamp: "Formatted at 2026-08-06 18:46:27 on dist-test-slave-fgb3"
server_key: "702ddf623cb591e5f2160a9adb340ff6"
server_key_iv: "d0027ec09f83d865e8a5df038b42a33d"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:27.751578 18461 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 18:46:27.776856 18482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.778755 18461 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 18:46:27.778909 18461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "332afd749ddf4a0bac2d547923cb80e9"
format_stamp: "Formatted at 2026-08-06 18:46:27 on dist-test-slave-fgb3"
server_key: "702ddf623cb591e5f2160a9adb340ff6"
server_key_iv: "d0027ec09f83d865e8a5df038b42a33d"
server_key_version: "kuduclusterkey@0"
I20260806 18:46:27.779072 18461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:46:27.819751 18461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:46:27.824040 18461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:46:27.824263 18461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:46:27.825143 18461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:46:27.826535 18461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:46:27.826617 18461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.826682 18461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:46:27.826714 18461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:46:27.849656 18461 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:35077
I20260806 18:46:27.851001 18461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:46:27.852020 18664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:35077 every 8 connection(s)
I20260806 18:46:27.854842 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 18461
I20260806 18:46:27.854974 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:46:27.855293 2005 external_mini_cluster.cc:1428] Setting key 5a07f548169fbbcfd83c20b0f11e25dc
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:46:27.871165 18249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.855890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:53995 (local address 127.1.245.126:40449)
0806 18:46:27.856016 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:27.856021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:27.856522 (+ 501us) server_negotiation.cc:407] Connection header received
0806 18:46:27.858016 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:27.858020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:27.858074 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:27.858123 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:46:27.860180 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.860982 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.862760 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.863121 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.863197 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.866832 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:46:27.866836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:46:27.866839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:46:27.866870 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:46:27.868584 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:27.869222 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:27.869226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:27.869227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:27.869317 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:27.869731 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:27.869735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:27.869737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:27.869987 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:46:27.870066 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:27.870774 (+ 708us) server_negotiation.cc:299] Negotiation successful
0806 18:46:27.870906 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:46:27.872852 18670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.854674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40449 (local address 127.1.245.67:53995)
0806 18:46:27.856364 (+ 1690us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:27.856427 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:27.857742 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:27.858580 (+ 838us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:46:27.858590 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:46:27.859136 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:46:27.860014 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.860037 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.861140 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.861144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.862623 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.862635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.863407 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.863411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.863703 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.864388 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:46:27.864409 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:46:27.866614 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:46:27.868730 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:46:27.868736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:46:27.868753 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:46:27.869110 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:46:27.869420 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:46:27.869423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:46:27.869424 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:46:27.869616 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:46:27.870134 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:46:27.870141 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:46:27.870499 (+ 358us) client_negotiation.cc:769] Sending connection context
0806 18:46:27.871408 (+ 909us) client_negotiation.cc:240] Negotiation successful
0806 18:46:27.871764 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1448,"thread_start_us":192,"threads_started":1}
Aug 06 18:46:27 dist-test-slave-fgb3 krb5kdc[17196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041987, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:46:27.874863 18667 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40449
I20260806 18:46:27.875133 18667 heartbeater.cc:464] Registering TS with master...
I20260806 18:46:27.875782 18667 heartbeater.cc:511] Master 127.1.245.126:40449 requested a full tablet report, sending...
I20260806 18:46:27.877005 17916 ts_manager.cc:194] Registered new tserver with Master: 332afd749ddf4a0bac2d547923cb80e9 (127.1.245.67:35077)
I20260806 18:46:27.877777 17916 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:53995
I20260806 18:46:27.881778 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:46:27.894506 18249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.884163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.1.245.126:40449)
0806 18:46:27.884293 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:27.884298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:27.885068 (+ 770us) server_negotiation.cc:407] Connection header received
0806 18:46:27.885281 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:27.885285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:27.885356 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:27.885424 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:46:27.886578 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.887403 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.889026 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.889335 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.889405 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.891322 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:46:27.891326 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:46:27.891330 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:46:27.891358 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:46:27.892847 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:27.893284 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:27.893297 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:27.893299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:27.893389 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:46:27.893639 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:46:27.893643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:46:27.893644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:46:27.893831 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:46:27.893946 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:27.894186 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 18:46:27.894336 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:46:27.897396 17916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54702:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:46:27.900697 17916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:46:27.929708 18683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.919684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:43471 (local address 127.0.0.1:35574)
0806 18:46:27.922050 (+ 2366us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:27.922063 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:27.922179 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:27.923169 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:46:27.923172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:46:27.923194 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:46:27.923428 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.923440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.924999 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.925003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.927690 (+ 2687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.927700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.929098 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.929102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.929271 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.929297 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:46:27.929385 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 18:46:27.929536 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2302,"mutex_wait_us":1277,"thread_start_us":203,"threads_started":1}
I20260806 18:46:27.930197 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.919229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:36895 (local address 127.0.0.1:37604)
0806 18:46:27.921632 (+ 2403us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:27.921645 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:27.921780 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:27.922403 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:46:27.922409 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:46:27.922434 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:46:27.923386 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.923397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.925046 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.925050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.927601 (+ 2551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.927612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.929104 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.929107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.929218 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.929274 (+ 56us) client_negotiation.cc:769] Sending connection context
0806 18:46:27.929385 (+ 111us) client_negotiation.cc:240] Negotiation successful
0806 18:46:27.930061 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2338,"mutex_wait_us":594,"thread_start_us":173,"threads_started":1}
I20260806 18:46:27.930603 18689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.920386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37604 (local address 127.1.245.66:36895)
0806 18:46:27.921417 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:27.921460 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:27.921775 (+ 315us) server_negotiation.cc:407] Connection header received
0806 18:46:27.921903 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:27.921909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:27.922122 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:27.923480 (+ 1358us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:46:27.923648 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.924907 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.927838 (+ 2931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.928800 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.928904 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.929043 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:27.929517 (+ 474us) server_negotiation.cc:299] Negotiation successful
0806 18:46:27.929619 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":945,"thread_start_us":69,"threads_started":1}
I20260806 18:46:27.933355 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.920693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35574 (local address 127.1.245.65:43471)
0806 18:46:27.920998 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:27.921006 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:27.922308 (+ 1302us) server_negotiation.cc:407] Connection header received
0806 18:46:27.922414 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:27.922420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:27.922941 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:27.923575 (+ 634us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:46:27.923720 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.924868 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.927843 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.928732 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.928864 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.929029 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:27.933060 (+ 4031us) server_negotiation.cc:299] Negotiation successful
0806 18:46:27.933178 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":120,"threads_started":1}
I20260806 18:46:27.934253 18345 tablet_service.cc:1467] Processing CreateTablet for tablet a42199da67a24649801f89bbd15c2425 (DEFAULT_TABLE table=test-table [id=e273f7a16dfc46109ba49bf8ee86c540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:46:27.936573 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.918854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35077 (local address 127.0.0.1:50134)
0806 18:46:27.920262 (+ 1408us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:27.920329 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:27.920462 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:27.927016 (+ 6554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:46:27.927019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:46:27.927040 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:46:27.927274 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.927285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.933554 (+ 6269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.933558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.934727 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:27.934741 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.936122 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.936126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:27.936270 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.936303 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 18:46:27.936359 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 18:46:27.936435 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1167,"thread_start_us":120,"threads_started":1}
I20260806 18:46:27.937361 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:27.919769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50134 (local address 127.1.245.67:35077)
0806 18:46:27.925614 (+ 5845us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:27.925660 (+ 46us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:27.925686 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:46:27.925768 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:27.925773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:27.925995 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:27.930524 (+ 4529us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:46:27.931604 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.932842 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.935036 (+ 2194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:27.935974 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:27.936873 (+ 899us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:27.937028 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:27.937081 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 18:46:27.937172 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5724,"thread_start_us":148,"threads_started":1}
I20260806 18:46:27.940445 18524 tablet_service.cc:1467] Processing CreateTablet for tablet a42199da67a24649801f89bbd15c2425 (DEFAULT_TABLE table=test-table [id=e273f7a16dfc46109ba49bf8ee86c540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:46:27.940740 18108 tablet_service.cc:1467] Processing CreateTablet for tablet a42199da67a24649801f89bbd15c2425 (DEFAULT_TABLE table=test-table [id=e273f7a16dfc46109ba49bf8ee86c540]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:46:27.941646 18524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42199da67a24649801f89bbd15c2425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:46:27.941886 18108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42199da67a24649801f89bbd15c2425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:46:27.946548 18345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a42199da67a24649801f89bbd15c2425. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:46:27.961897 18695 tablet_bootstrap.cc:492] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Bootstrap starting.
I20260806 18:46:27.964665 18695 tablet_bootstrap.cc:654] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Neither blocks nor log segments found. Creating new log.
I20260806 18:46:27.965338 18697 tablet_bootstrap.cc:492] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843: Bootstrap starting.
I20260806 18:46:27.966352 18695 log.cc:824] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:46:27.966856 18693 tablet_bootstrap.cc:492] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Bootstrap starting.
I20260806 18:46:27.968689 18697 tablet_bootstrap.cc:654] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843: Neither blocks nor log segments found. Creating new log.
I20260806 18:46:27.969961 18697 log.cc:824] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843: Log is configured to *not* fsync() on all Append() calls
I20260806 18:46:27.970463 18693 tablet_bootstrap.cc:654] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Neither blocks nor log segments found. Creating new log.
I20260806 18:46:27.972033 18693 log.cc:824] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:46:27.972216 18695 tablet_bootstrap.cc:492] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: No bootstrap required, opened a new log
I20260806 18:46:27.972491 18695 ts_tablet_manager.cc:1397] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260806 18:46:27.979929 18693 tablet_bootstrap.cc:492] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: No bootstrap required, opened a new log
I20260806 18:46:27.980178 18693 ts_tablet_manager.cc:1397] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.006s
I20260806 18:46:27.980597 18697 tablet_bootstrap.cc:492] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843: No bootstrap required, opened a new log
I20260806 18:46:27.980868 18697 ts_tablet_manager.cc:1397] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.007s
I20260806 18:46:27.986258 18693 raft_consensus.cc:348] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:27.986629 18693 raft_consensus.cc:374] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:46:27.986686 18693 raft_consensus.cc:729] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 332afd749ddf4a0bac2d547923cb80e9, State: Initialized, Role: FOLLOWER
I20260806 18:46:27.987303 18693 consensus_queue.cc:260] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:27.986115 18695 raft_consensus.cc:348] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:27.987883 18695 raft_consensus.cc:374] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:46:27.988528 18693 ts_tablet_manager.cc:1428] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260806 18:46:27.988682 18695 raft_consensus.cc:729] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f70e36897dcd4537976365f25d7026b9, State: Initialized, Role: FOLLOWER
I20260806 18:46:27.988960 18667 heartbeater.cc:503] Master 127.1.245.126:40449 was elected leader, sending a full tablet report...
I20260806 18:46:27.990170 18697 raft_consensus.cc:348] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:27.990492 18697 raft_consensus.cc:374] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:46:27.990543 18697 raft_consensus.cc:729] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5784053dd6348d1932ea2807ec59843, State: Initialized, Role: FOLLOWER
I20260806 18:46:27.991091 18697 consensus_queue.cc:260] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:27.992379 18697 ts_tablet_manager.cc:1428] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843: Time spent starting tablet: real 0.011s user 0.007s sys 0.001s
I20260806 18:46:27.993330 18246 heartbeater.cc:503] Master 127.1.245.126:40449 was elected leader, sending a full tablet report...
I20260806 18:46:27.994796 18695 consensus_queue.cc:260] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:27.996153 18695 ts_tablet_manager.cc:1428] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Time spent starting tablet: real 0.024s user 0.008s sys 0.000s
I20260806 18:46:27.996587 18450 heartbeater.cc:503] Master 127.1.245.126:40449 was elected leader, sending a full tablet report...
W20260806 18:46:28.075947 18247 tablet.cc:2369] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:46:28.086244 18703 raft_consensus.cc:482] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:46:28.086642 18703 raft_consensus.cc:504] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:28.088169 18703 leader_election.cc:288] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 332afd749ddf4a0bac2d547923cb80e9 (127.1.245.67:35077), f5784053dd6348d1932ea2807ec59843 (127.1.245.65:43471)
I20260806 18:46:28.099668 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:28.089426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:50859 (local address 127.1.245.65:43471)
0806 18:46:28.089560 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:28.089565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:28.090881 (+ 1316us) server_negotiation.cc:407] Connection header received
0806 18:46:28.091135 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:28.091139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:28.091188 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:28.091236 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:46:28.091872 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.092770 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.094758 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.098756 (+ 3998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.098868 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:28.098923 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:28.099444 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 18:46:28.099534 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:46:28.100235 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:28.088860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:43471 (local address 127.1.245.66:50859)
0806 18:46:28.090738 (+ 1878us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:28.090753 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:28.090969 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:28.091405 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:46:28.091407 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:46:28.091429 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:46:28.091726 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:28.091737 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.092925 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.092929 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:28.094605 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:28.094622 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.099105 (+ 4483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.099109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:28.099288 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:28.099315 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:46:28.099832 (+ 517us) client_negotiation.cc:240] Negotiation successful
0806 18:46:28.099926 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1802,"thread_start_us":206,"threads_started":1}
W20260806 18:46:28.104244 18669 tablet.cc:2369] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:46:28.104628 18686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:28.090151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:50111 (local address 127.1.245.67:35077)
0806 18:46:28.090324 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:28.090330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:28.095843 (+ 5513us) server_negotiation.cc:407] Connection header received
0806 18:46:28.096087 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:28.096091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:28.096178 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:28.096237 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:46:28.100508 (+ 4271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.101267 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.103114 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.103773 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.103889 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:28.103941 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:28.104425 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 18:46:28.104513 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:46:28.105053 18706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:28.089683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35077 (local address 127.1.245.66:50111)
0806 18:46:28.095703 (+ 6020us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:28.095719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:28.095937 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:28.096402 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:46:28.096406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:46:28.096428 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:46:28.096826 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:28.096837 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.101554 (+ 4717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.101557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:28.102956 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:46:28.102969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.104099 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.104103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:46:28.104251 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:28.104290 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 18:46:28.104764 (+ 474us) client_negotiation.cc:240] Negotiation successful
0806 18:46:28.104851 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5931,"thread_start_us":180,"threads_started":1}
I20260806 18:46:28.106033 18586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42199da67a24649801f89bbd15c2425" candidate_uuid: "f70e36897dcd4537976365f25d7026b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "332afd749ddf4a0bac2d547923cb80e9" is_pre_election: true
I20260806 18:46:28.106253 18168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42199da67a24649801f89bbd15c2425" candidate_uuid: "f70e36897dcd4537976365f25d7026b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5784053dd6348d1932ea2807ec59843" is_pre_election: true
I20260806 18:46:28.106621 18168 raft_consensus.cc:2454] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f70e36897dcd4537976365f25d7026b9 in term 0.
I20260806 18:46:28.106808 18586 raft_consensus.cc:2454] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f70e36897dcd4537976365f25d7026b9 in term 0.
I20260806 18:46:28.107306 18291 leader_election.cc:302] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5784053dd6348d1932ea2807ec59843, f70e36897dcd4537976365f25d7026b9; no voters:
I20260806 18:46:28.107739 18703 raft_consensus.cc:2781] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:46:28.107837 18703 raft_consensus.cc:482] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:46:28.107878 18703 raft_consensus.cc:3037] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:46:28.109956 18703 raft_consensus.cc:504] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:28.110402 18703 leader_election.cc:288] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [CANDIDATE]: Term 1 election: Requested vote from peers 332afd749ddf4a0bac2d547923cb80e9 (127.1.245.67:35077), f5784053dd6348d1932ea2807ec59843 (127.1.245.65:43471)
I20260806 18:46:28.110874 18168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42199da67a24649801f89bbd15c2425" candidate_uuid: "f70e36897dcd4537976365f25d7026b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5784053dd6348d1932ea2807ec59843"
I20260806 18:46:28.110996 18168 raft_consensus.cc:3037] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:46:28.112314 18168 raft_consensus.cc:2454] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f70e36897dcd4537976365f25d7026b9 in term 1.
I20260806 18:46:28.112701 18291 leader_election.cc:302] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f5784053dd6348d1932ea2807ec59843, f70e36897dcd4537976365f25d7026b9; no voters:
I20260806 18:46:28.113354 18586 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a42199da67a24649801f89bbd15c2425" candidate_uuid: "f70e36897dcd4537976365f25d7026b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "332afd749ddf4a0bac2d547923cb80e9"
I20260806 18:46:28.113494 18586 raft_consensus.cc:3037] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:46:28.114856 18586 raft_consensus.cc:2454] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f70e36897dcd4537976365f25d7026b9 in term 1.
I20260806 18:46:28.115512 18703 raft_consensus.cc:2781] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:46:28.115754 18703 raft_consensus.cc:686] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 1 LEADER]: Becoming Leader. State: Replica: f70e36897dcd4537976365f25d7026b9, State: Running, Role: LEADER
I20260806 18:46:28.116304 18703 consensus_queue.cc:237] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } }
I20260806 18:46:28.132515 17916 catalog_manager.cc:5495] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f70e36897dcd4537976365f25d7026b9 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "f70e36897dcd4537976365f25d7026b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f70e36897dcd4537976365f25d7026b9" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 } health_report { overall_health: UNKNOWN } } }
I20260806 18:46:28.185073 18689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:28.180230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.1.245.66:36895)
0806 18:46:28.180385 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0806 18:46:28.180390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:46:28.180493 (+ 103us) server_negotiation.cc:407] Connection header received
0806 18:46:28.180683 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:46:28.180686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:46:28.180770 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:46:28.180899 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:46:28.181177 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.181939 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.183275 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:46:28.183616 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:46:28.183724 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:46:28.184082 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:46:28.184657 (+ 575us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:46:28.184790 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 18:46:28.184854 (+ 64us) server_negotiation.cc:299] Negotiation successful
0806 18:46:28.184923 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260806 18:46:28.190416 18454 tablet.cc:2369] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:46:28.194689 18168 raft_consensus.cc:1261] T a42199da67a24649801f89bbd15c2425 P f5784053dd6348d1932ea2807ec59843 [term 1 FOLLOWER]: Refusing update from remote peer f70e36897dcd4537976365f25d7026b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:46:28.195044 18586 raft_consensus.cc:1261] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 1 FOLLOWER]: Refusing update from remote peer f70e36897dcd4537976365f25d7026b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:46:28.195887 18708 consensus_queue.cc:1035] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5784053dd6348d1932ea2807ec59843" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 43471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:46:28.196095 18703 consensus_queue.cc:1035] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "332afd749ddf4a0bac2d547923cb80e9" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35077 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:46:28.217238 18713 mvcc.cc:205] Tried to move back new op lower bound from 7315627983636443136 to 7315627983328575488. Current Snapshot: MvccSnapshot[applied={T|T < 7315627983636443136}]
I20260806 18:46:28.225063 18714 mvcc.cc:205] Tried to move back new op lower bound from 7315627983636443136 to 7315627983328575488. Current Snapshot: MvccSnapshot[applied={T|T < 7315627983636443136}]
I20260806 18:46:28.226807 18715 mvcc.cc:205] Tried to move back new op lower bound from 7315627983636443136 to 7315627983328575488. Current Snapshot: MvccSnapshot[applied={T|T < 7315627983636443136}]
I20260806 18:46:28.238277 17914 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:46:28.238533 17914 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54702:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:46:28.241693 17914 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 363f9cec88874f7aac0f93b5225f06f0: Sending DeleteTablet for 3 replicas of tablet a42199da67a24649801f89bbd15c2425
I20260806 18:46:28.243351 18108 tablet_service.cc:1514] Processing DeleteTablet for tablet a42199da67a24649801f89bbd15c2425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:46:28 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35574
I20260806 18:46:28.243999 18345 tablet_service.cc:1514] Processing DeleteTablet for tablet a42199da67a24649801f89bbd15c2425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:46:28 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:37604
I20260806 18:46:28.244942 18524 tablet_service.cc:1514] Processing DeleteTablet for tablet a42199da67a24649801f89bbd15c2425 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:46:28 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:50134
I20260806 18:46:28.246270 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 18035
I20260806 18:46:28.258354 18732 tablet_replica.cc:330] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: stopping tablet replica
I20260806 18:46:28.258884 18732 raft_consensus.cc:2229] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:46:28.259493 18732 raft_consensus.cc:2258] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:46:28.265982 18732 ts_tablet_manager.cc:1905] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:46:28.266556 18731 tablet_replica.cc:330] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: stopping tablet replica
I20260806 18:46:28.266986 18731 raft_consensus.cc:2229] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:46:28.267521 18731 raft_consensus.cc:2258] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:46:28.269217 18731 ts_tablet_manager.cc:1905] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:46:28.269277 18732 ts_tablet_manager.cc:1918] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:46:28.269409 18732 log.cc:1196] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/a42199da67a24649801f89bbd15c2425
I20260806 18:46:28.270116 18732 ts_tablet_manager.cc:1939] T a42199da67a24649801f89bbd15c2425 P 332afd749ddf4a0bac2d547923cb80e9: Deleting consensus metadata
W20260806 18:46:28.270697 17893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:43471 (error 108)
I20260806 18:46:28.272184 18731 ts_tablet_manager.cc:1918] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:46:28.272369 18731 log.cc:1196] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/a42199da67a24649801f89bbd15c2425
I20260806 18:46:28.272747 18731 ts_tablet_manager.cc:1939] T a42199da67a24649801f89bbd15c2425 P f70e36897dcd4537976365f25d7026b9: Deleting consensus metadata
I20260806 18:46:28.273057 17891 catalog_manager.cc:4841] TS 332afd749ddf4a0bac2d547923cb80e9 (127.1.245.67:35077): tablet a42199da67a24649801f89bbd15c2425 (table test-table [id=e273f7a16dfc46109ba49bf8ee86c540]) successfully deleted
I20260806 18:46:28.273406 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 18255
I20260806 18:46:28.292325 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:46:28.291856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:43471 (local address 127.0.0.1:35578)
0806 18:46:28.292119 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:28.292215 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:43471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
W20260806 18:46:28.293190 17893 catalog_manager.cc:4584] TS f5784053dd6348d1932ea2807ec59843 (127.1.245.65:43471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a42199da67a24649801f89bbd15c2425: Network error: Client connection negotiation failed: client connection to 127.1.245.65:43471: connect: Connection refused (error 111)
W20260806 18:46:28.300908 18685 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:46:28.300237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:36895 (local address 127.0.0.1:37626)
0806 18:46:28.300444 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0806 18:46:28.300459 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:46:28.300559 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:46:28.300761 (+ 202us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.66:36895: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":121}
I20260806 18:46:28.301689 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 18461
I20260806 18:46:28.320909 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 17860
I20260806 18:46:28.338956 2005 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:46:28.683900 2005 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:46:28.684027 2005 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:46:29.028797 2005 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:46:29.029 UTC [17241] LOG: received smart shutdown request
2026-08-06 18:46:29.033 UTC [17241] LOG: background worker "logical replication launcher" (PID 17253) exited with exit code 1
2026-08-06 18:46:29.033 UTC [17247] LOG: shutting down
2026-08-06 18:46:29.200 UTC [17241] LOG: database system is shut down
2026-08-06T18:46:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49462 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:46:29 dist-test-slave-fgb3 krb5kdc[18747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:46:29 dist-test-slave-fgb3 krb5kdc[18747](info): set up 2 sockets
Aug 06 18:46:29 dist-test-slave-fgb3 krb5kdc[18747](info): commencing operation
krb5kdc: starting...
W20260806 18:46:30.448195 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:46:30 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786041990, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:46:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:46:30Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.245.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.245.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/rangeradmin_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/rangeradmin_127.1.245.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.245.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.245.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/rangerlookup_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/rangerlookup_127.1.245.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.245.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.245.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/HTTP_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/HTTP_127.1.245.84@KRBTEST.COM.keytab.
I20260806 18:46:30.625530 2005 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskesmHnt/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/postgres -l logfile start
2026-08-06 18:46:33.017 UTC [18794] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 18:46:33.017 UTC [18794] LOG: listening on IPv4 address "127.1.245.84", port 39185
2026-08-06 18:46:33.020 UTC [18794] LOG: listening on Unix socket "/tmp/.s.PGSQL.39185"
2026-08-06 18:46:33.041 UTC [18797] LOG: database system was shut down at 2026-08-06 18:46:31 UTC
2026-08-06 18:46:33.045 UTC [18794] LOG: database system is ready to accept connections
I20260806 18:46:34.069314 2005 mini_postgres.cc:96] Postgres bound to 39185
2026-08-06 18:46:34.076 UTC [18806] FATAL: database "slave" does not exist
127.1.245.84:39185 - accepting connections
I20260806 18:46:34.076507 2005 mini_ranger.cc:161] Starting Ranger...
I20260806 18:46:34.091766 2005 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 18:46:34.533308 2005 mini_ranger.cc:87] Created ranger Postgres database
I20260806 18:46:34.533380 2005 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-admin
2026-08-06 18:46:34,807 [I] DB FLAVOR :POSTGRES
2026-08-06 18:46:34,808 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:46:34,808 [I] Checking connection..
2026-08-06 18:46:34,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 18:46:35,056 [I] Checking connection passed.
2026-08-06 18:46:35,056 [I] --------- Verifying version history table ---------
2026-08-06 18:46:35,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:46:35,319 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:46:35,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:46:35,576 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 18:46:35,576 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 18:46:35,576 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 18:46:35.825 UTC [18895] WARNING: there is no transaction in progress
2026-08-06 18:46:35,835 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 18:46:35,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 18:46:36,101 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 18:46:36,101 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:46:36,102 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 18:46:36,369 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 18:46:36,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-fgb3.c.gcp-upstream.internal','N') ;"
2026-08-06 18:46:36,850 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:46:37,087 [I] Table x_portal_user does not exist in database ranger
2026-08-06 18:46:37,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:46:37,361 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 18:46:37,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:46:37,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:46:37,833 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 18:46:37,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 18:46:38.834 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:38.850 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:38.867 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:38.882 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:38.898 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:38.915 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:38.969 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:38.977 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:38.992 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:39.003 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:39.014 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:39.028 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:39.038 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:39.045 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:39.052 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:39.059 UTC [19052] WARNING: there is no transaction in progress
2026-08-06 18:46:39,703 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 18:46:39,703 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 18:46:39,969 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 18:46:39,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 18:46:40,233 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 18:46:40,233 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 18:46:40,475 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 18:46:40,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 18:46:40,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 18:46:40,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-fgb3.c.gcp-upstream.internal';"
2026-08-06 18:46:41,246 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 18:46:41,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 18:46:41,497 [I] DB_PATCHES have already been applied
I20260806 18:46:41.503659 2005 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-admin:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskesmHnt/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 18:46:41.503731 2005 mini_ranger.cc:193] Using host: 127.1.245.84
I20260806 18:46:41.507798 2005 mini_ranger.cc:227] Ranger admin URL: http://127.1.245.84:40383
Aug 06, 2026 6:46:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:46:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:46:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 6:46:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 6:46:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 6:46:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 6:46:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 18:46:42.675531 2005 mini_ranger.cc:160] Time spent starting Ranger: real 8.599s user 0.002s sys 0.007s
Aug 06, 2026 6:47:02 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 6:47:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 06, 2026 6:47:02 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 6:47:02 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 6:47:03 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 18:47:03.893437 2005 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.245.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.245.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/rangerkms_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.245.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/rangerkms_127.1.245.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.245.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.245.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.245.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/HTTP_127.1.245.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.245.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/HTTP_127.1.245.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 18:47:03 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042023, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 18:47:03.984236 2005 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 18:47:03.994169 2005 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 18:47:04.786489 2005 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 18:47:04.786573 2005 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms
I20260806 18:47:04.786592 2005 mini_ranger_kms.cc:227] Using postgres at 127.1.245.84:39185
2026-08-06 18:47:04,955 [I] DB FLAVOR :POSTGRES
2026-08-06 18:47:04,956 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 18:47:04,956 [I] Checking connection
2026-08-06 18:47:04,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 18:47:05,189 [I] connection success
2026-08-06 18:47:05,189 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 18:47:05,189 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 18:47:05,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 18:47:05,443 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 18:47:05,444 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 18:47:05,444 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 18:47:05,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.245.84:39185/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 18:47:05,707 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 18:47:06.138856 2005 mini_ranger_kms.cc:310] Created kms service
I20260806 18:47:06.315209 2005 mini_ranger_kms.cc:326] Created kudu user
I20260806 18:47:06.361075 2005 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 18:47:06 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042023, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:06.756771 2005 mini_ranger_kms.cc:384] Added ranger policy
I20260806 18:47:06.756932 2005 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms:/tmp/dist-test-taskesmHnt/build/debug/bin/postgresql.jar:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskesmHnt/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 18:47:06.756950 2005 mini_ranger_kms.cc:240] Using host: 127.1.245.84
I20260806 18:47:06.761670 2005 mini_ranger_kms.cc:276] Ranger KMS PID: 19323
I20260806 18:47:06.761759 2005 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.245.84:55741
Aug 06, 2026 6:47:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 6:47:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 6:47:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 6:47:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 6:47:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 6:47:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 18:47:09 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786042029, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.245.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:09.204520 2005 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.220s user 0.006s sys 0.149s
I20260806 18:47:09.204655 2005 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 18:47:09.204720 2005 mini_ranger_kms.cc:401] 127.1.245.84:55741/kms/v1/keys
Aug 06 18:47:09 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042029, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.245.84@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
Aug 06 18:47:09 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042029, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.245.84@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
Aug 06 18:47:12 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:47:12.099370 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:43201
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:43201
--ranger_config_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:55741/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:12.216324 19411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:12.216599 19411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:12.216638 19411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:12.220640 19411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:12.220711 19411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:12.220729 19411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:12.220744 19411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:12.220757 19411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:12.226447 19411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.245.84:55741/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:43201
--ranger_config_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:43201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:12.227596 19411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:12.228654 19411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:12.237432 19419 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:12.237419 19416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:12.237579 19411 server_base.cc:1034] running on GCE node
W20260806 18:47:12.237437 19417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:12.238230 19411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:12.239435 19411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:12.240613 19411 hybrid_clock.cc:648] HybridClock initialized: now 1786042032240590 us; error 33 us; skew 500 ppm
Aug 06 18:47:12 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042032, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:12.243373 19411 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:47:12.244233 19411 webserver.cc:460] Webserver started at http://127.1.245.126:35151/ using document root <none> and password file <none>
I20260806 18:47:12.244762 19411 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:12.244822 19411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:12.245018 19411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:47:12 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042032, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:12.291636 19411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "6bd7538e4c28470083b49f5c07d89759"
format_stamp: "Formatted at 2026-08-06 18:47:12 on dist-test-slave-fgb3"
server_key: "2d9f81ccd884f7a9717cb6df304a1ff8"
server_key_iv: "23e97c2a139eb31fe692524bf2d76371"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:12.292472 19411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "6bd7538e4c28470083b49f5c07d89759"
format_stamp: "Formatted at 2026-08-06 18:47:12 on dist-test-slave-fgb3"
server_key: "2d9f81ccd884f7a9717cb6df304a1ff8"
server_key_iv: "23e97c2a139eb31fe692524bf2d76371"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:12.297739 19411 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260806 18:47:12.315486 19431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:12.317198 19411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:47:12.317350 19411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "6bd7538e4c28470083b49f5c07d89759"
format_stamp: "Formatted at 2026-08-06 18:47:12 on dist-test-slave-fgb3"
server_key: "2d9f81ccd884f7a9717cb6df304a1ff8"
server_key_iv: "23e97c2a139eb31fe692524bf2d76371"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:12.317490 19411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:12.349617 19411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:12.353047 19411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:12.353266 19411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:12.362011 19411 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:43201
I20260806 18:47:12.362028 19511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:43201 every 8 connection(s)
I20260806 18:47:12.363221 19411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:47:12.366106 19512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:12.367846 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 19411
I20260806 18:47:12.367981 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:47:12.368290 2005 external_mini_cluster.cc:1428] Setting key 07b5abe6f2aedd835b569cf51a6035d2
I20260806 18:47:12.372921 19512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Bootstrap starting.
I20260806 18:47:12.375396 19512 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:12.376330 19512 log.cc:824] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:12.378625 19512 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: No bootstrap required, opened a new log
Aug 06 18:47:12 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:12.382833 19512 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } }
I20260806 18:47:12.383109 19512 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:12.383188 19512 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd7538e4c28470083b49f5c07d89759, State: Initialized, Role: FOLLOWER
I20260806 18:47:12.383671 19512 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } }
I20260806 18:47:12.383875 19512 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:47:12.383960 19512 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:47:12.384081 19512 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:12.384820 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:12.369790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43210 (local address 127.1.245.126:43201)
0806 18:47:12.370249 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:12.370269 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:12.370333 (+ 64us) server_negotiation.cc:407] Connection header received
0806 18:47:12.371116 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:12.371141 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:12.371580 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:12.371946 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:12.372793 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:12.374213 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:12.375479 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:12.376155 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:12.376326 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:12.379331 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:12.379337 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:12.379353 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:12.379384 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:12.381329 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:12.381819 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:12.381825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:12.381831 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:12.381964 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:12.382246 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:12.382249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:12.382251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:12.382782 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:12.382993 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:12.383594 (+ 601us) server_negotiation.cc:299] Negotiation successful
0806 18:47:12.383868 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":127,"threads_started":1}
I20260806 18:47:12.385253 19512 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } }
I20260806 18:47:12.385610 19512 leader_election.cc:302] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bd7538e4c28470083b49f5c07d89759; no voters:
I20260806 18:47:12.385928 19512 leader_election.cc:288] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:47:12.387215 19521 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:12.387477 19521 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 1 LEADER]: Becoming Leader. State: Replica: 6bd7538e4c28470083b49f5c07d89759, State: Running, Role: LEADER
I20260806 18:47:12.387275 19512 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:12.387857 19521 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } }
I20260806 18:47:12.389083 19521 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bd7538e4c28470083b49f5c07d89759. Latest consensus state: current_term: 1 leader_uuid: "6bd7538e4c28470083b49f5c07d89759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } } }
I20260806 18:47:12.389284 19521 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:12.389521 19522 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bd7538e4c28470083b49f5c07d89759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } } }
I20260806 18:47:12.389631 19522 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:12.389817 19541 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:47:12.392931 19541 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:47:12.389832 19512 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:47:12.403486 19541 catalog_manager.cc:1347] Generated new cluster ID: ed29969d93db487da33f03ab6b817ae1
I20260806 18:47:12.403553 19541 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:47:12.417713 19541 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:47:12.419092 19541 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:47:12.428594 19541 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Generated new TSK 0
I20260806 18:47:12.429548 19541 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:47:13.119387 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:55741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:43201
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:13.239629 19574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:13.239957 19574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:13.240010 19574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:13.244081 19574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:13.244138 19574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:13.244199 19574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:47:13.251060 19574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.245.84:55741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:13.252744 19574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:13.254143 19574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:13.265910 19582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:13.266618 19574 server_base.cc:1034] running on GCE node
W20260806 18:47:13.266691 19584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:13.266861 19588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:13.267612 19574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:13.268479 19574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:13.269729 19574 hybrid_clock.cc:648] HybridClock initialized: now 1786042033269694 us; error 48 us; skew 500 ppm
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:13.273428 19574 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:47:13.274667 19574 webserver.cc:460] Webserver started at http://127.1.245.65:38387/ using document root <none> and password file <none>
I20260806 18:47:13.275410 19574 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:13.275507 19574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:13.275842 19574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:13.294248 19574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "535ca53409da4be198f8edafa8df7634"
format_stamp: "Formatted at 2026-08-06 18:47:13 on dist-test-slave-fgb3"
server_key: "6e8571341d2af47b2857867c5d9e753e"
server_key_iv: "b8d4661af7d6ff6d9d2e297e5b990b5d"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:13.295593 19574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "535ca53409da4be198f8edafa8df7634"
format_stamp: "Formatted at 2026-08-06 18:47:13 on dist-test-slave-fgb3"
server_key: "6e8571341d2af47b2857867c5d9e753e"
server_key_iv: "b8d4661af7d6ff6d9d2e297e5b990b5d"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:13.301190 19574 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:47:13.346710 19597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:13.348577 19574 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.000s
I20260806 18:47:13.348735 19574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "535ca53409da4be198f8edafa8df7634"
format_stamp: "Formatted at 2026-08-06 18:47:13 on dist-test-slave-fgb3"
server_key: "6e8571341d2af47b2857867c5d9e753e"
server_key_iv: "b8d4661af7d6ff6d9d2e297e5b990b5d"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:13.348901 19574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:13.375190 19574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:13.378669 19574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:13.378922 19574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:13.379645 19574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:13.380759 19574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:13.380826 19574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:13.380890 19574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:13.380920 19574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:13.392211 19574 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:36615
I20260806 18:47:13.394477 19574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:47:13.397303 19768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:36615 every 8 connection(s)
I20260806 18:47:13.400307 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 19574
I20260806 18:47:13.400432 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:47:13.400724 2005 external_mini_cluster.cc:1428] Setting key 44af5b1e3700de51027dac5677b45f14
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:13.423593 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:13.400977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:49629 (local address 127.1.245.126:43201)
0806 18:47:13.404127 (+ 3150us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:13.404138 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:13.405403 (+ 1265us) server_negotiation.cc:407] Connection header received
0806 18:47:13.405655 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:13.405658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:13.405711 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:13.405791 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:13.407305 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:13.408166 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:13.409855 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:13.410225 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:13.410308 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:13.417829 (+ 7521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:13.417835 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:13.417838 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:13.417875 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:13.419906 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:13.420728 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:13.420732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:13.420734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:13.420845 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:13.421327 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:13.421331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:13.421333 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:13.421598 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:13.421781 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:13.423102 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0806 18:47:13.423249 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1415,"thread_start_us":106,"threads_started":1}
I20260806 18:47:13.425503 19776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:13.399793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:43201 (local address 127.1.245.65:49629)
0806 18:47:13.404420 (+ 4627us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:13.404478 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:13.405468 (+ 990us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:13.406094 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:13.406103 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:13.406459 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:13.407131 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:13.407155 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:13.408333 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:13.408336 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:13.409717 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:13.409730 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:13.410549 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:13.410553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:13.410766 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:13.415741 (+ 4975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:13.415770 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:13.417560 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:13.420098 (+ 2538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:13.420105 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:13.420126 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:13.420511 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:13.420980 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:13.420984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:13.420986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:13.421193 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:13.421862 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:13.421869 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:13.422387 (+ 518us) client_negotiation.cc:769] Sending connection context
0806 18:47:13.424148 (+ 1761us) client_negotiation.cc:240] Negotiation successful
0806 18:47:13.424537 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4388,"thread_start_us":154,"threads_started":1}
I20260806 18:47:13.426941 19771 heartbeater.cc:347] Connected to a master server at 127.1.245.126:43201
I20260806 18:47:13.427290 19771 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:13.427874 19771 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:13.429714 19453 ts_manager.cc:194] Registered new tserver with Master: 535ca53409da4be198f8edafa8df7634 (127.1.245.65:36615)
I20260806 18:47:13.430886 19453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:49629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:47:13.433759 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:55741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:43201
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 18:47:13.562242 19789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:13.562565 19789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:13.562656 19789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:13.566951 19789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:13.567015 19789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:13.567096 19789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:47:13.572604 19789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.245.84:55741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:13.573796 19789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:13.574906 19789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:13.584517 19802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:13.584502 19799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:13.584926 19789 server_base.cc:1034] running on GCE node
W20260806 18:47:13.584721 19800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:13.585544 19789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:13.586395 19789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:13.587582 19789 hybrid_clock.cc:648] HybridClock initialized: now 1786042033587555 us; error 44 us; skew 500 ppm
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:13.591255 19789 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:47:13.592483 19789 webserver.cc:460] Webserver started at http://127.1.245.66:42387/ using document root <none> and password file <none>
I20260806 18:47:13.593284 19789 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:13.593425 19789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:13.593711 19789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:13.607828 19789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "7fc8afe4b5c143b293749920c0eb49bf"
format_stamp: "Formatted at 2026-08-06 18:47:13 on dist-test-slave-fgb3"
server_key: "a7a4705a52785d3d41dd10cf18cd5386"
server_key_iv: "11590733c3983cd7660cf9c1c543e36c"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:13.609107 19789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "7fc8afe4b5c143b293749920c0eb49bf"
format_stamp: "Formatted at 2026-08-06 18:47:13 on dist-test-slave-fgb3"
server_key: "a7a4705a52785d3d41dd10cf18cd5386"
server_key_iv: "11590733c3983cd7660cf9c1c543e36c"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:13.614697 19789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260806 18:47:13.642798 19811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:13.644692 19789 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.003s sys 0.001s
I20260806 18:47:13.644840 19789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "7fc8afe4b5c143b293749920c0eb49bf"
format_stamp: "Formatted at 2026-08-06 18:47:13 on dist-test-slave-fgb3"
server_key: "a7a4705a52785d3d41dd10cf18cd5386"
server_key_iv: "11590733c3983cd7660cf9c1c543e36c"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:13.644977 19789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:13.689380 19789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:13.693562 19789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:13.693898 19789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:13.694808 19789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:13.696095 19789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:13.696233 19789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:13.696352 19789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:13.696429 19789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:13.712636 19789 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:33979
I20260806 18:47:13.713994 19789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:47:13.714946 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 19789
I20260806 18:47:13.715085 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:47:13.715451 2005 external_mini_cluster.cc:1428] Setting key 8d8e5a70785277176bf73ae532e779ac
I20260806 18:47:13.722718 19988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:33979 every 8 connection(s)
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:13.732923 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:13.718670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:33129 (local address 127.1.245.126:43201)
0806 18:47:13.718843 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:13.718848 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:13.719224 (+ 376us) server_negotiation.cc:407] Connection header received
0806 18:47:13.720546 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:13.720550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:13.720612 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:13.720677 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:13.722527 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:13.723332 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:13.725050 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:13.725303 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:13.725410 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:13.728160 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:13.728164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:13.728167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:13.728202 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:13.730022 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:13.730798 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:13.730802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:13.730804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:13.730911 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:13.731379 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:13.731384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:13.731385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:13.731647 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:13.731760 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:13.732558 (+ 798us) server_negotiation.cc:299] Negotiation successful
0806 18:47:13.732726 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:47:13.734766 19993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:13.717437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:43201 (local address 127.1.245.66:33129)
0806 18:47:13.719052 (+ 1615us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:13.719118 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:13.720271 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:13.721152 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:13.721162 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:13.721563 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:13.722297 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:13.722345 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:13.723546 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:13.723550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:13.724910 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:13.724921 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:13.725472 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:13.725478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:13.725663 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:13.726338 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:13.726364 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:13.727970 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:13.730180 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:13.730187 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:13.730207 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:13.730644 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:13.731040 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:13.731044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:13.731046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:13.731246 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:13.731840 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:13.731847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:13.732226 (+ 379us) client_negotiation.cc:769] Sending connection context
0806 18:47:13.733445 (+ 1219us) client_negotiation.cc:240] Negotiation successful
0806 18:47:13.733842 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1379,"thread_start_us":180,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
I20260806 18:47:13.736222 19990 heartbeater.cc:347] Connected to a master server at 127.1.245.126:43201
I20260806 18:47:13.736485 19990 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:13.737113 19990 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:13.738449 19453 ts_manager.cc:194] Registered new tserver with Master: 7fc8afe4b5c143b293749920c0eb49bf (127.1.245.66:33979)
I20260806 18:47:13.739231 19453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:33129
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:47:13.759974 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:55741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:43201
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:13.940881 19997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:13.941416 19997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:13.941568 19997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:13.947871 19997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:13.948089 19997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:13.948230 19997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:47:13.956804 19997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.245.84:55741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:13.958664 19997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:13.960175 19997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:47:13.974136 19997 server_base.cc:1034] running on GCE node
W20260806 18:47:13.975242 20004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:13.978750 20006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:13.982463 20003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:13.983224 19997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:13.984185 19997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:13.988376 19997 hybrid_clock.cc:648] HybridClock initialized: now 1786042033988287 us; error 90 us; skew 500 ppm
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:13.992594 19997 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:47:13.993932 19997 webserver.cc:460] Webserver started at http://127.1.245.67:37093/ using document root <none> and password file <none>
I20260806 18:47:13.994810 19997 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:13.994972 19997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:13.995286 19997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 18:47:13 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:14.013670 19997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "a9ae00555cce4898818ee52bb52a1932"
format_stamp: "Formatted at 2026-08-06 18:47:14 on dist-test-slave-fgb3"
server_key: "85279f689540cf231ecece8c593a417d"
server_key_iv: "11f35ed0602575752c891d5564626cd4"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:14.019011 19997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "a9ae00555cce4898818ee52bb52a1932"
format_stamp: "Formatted at 2026-08-06 18:47:14 on dist-test-slave-fgb3"
server_key: "85279f689540cf231ecece8c593a417d"
server_key_iv: "11f35ed0602575752c891d5564626cd4"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:14.029163 19997 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:14.057091 20021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:14.059105 19997 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.004s sys 0.000s
I20260806 18:47:14.059258 19997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "a9ae00555cce4898818ee52bb52a1932"
format_stamp: "Formatted at 2026-08-06 18:47:14 on dist-test-slave-fgb3"
server_key: "85279f689540cf231ecece8c593a417d"
server_key_iv: "11f35ed0602575752c891d5564626cd4"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:14.059396 19997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:14.090983 19997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:14.096441 19997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:14.096760 19997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:14.097589 19997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:14.098896 19997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:14.099009 19997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:14.099104 19997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:14.099162 19997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:14.109086 19997 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:33053
I20260806 18:47:14.110271 19997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:47:14.119149 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 19997
I20260806 18:47:14.119287 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:47:14.119590 2005 external_mini_cluster.cc:1428] Setting key af0db542bf6ae50934e4e4a673106b57
I20260806 18:47:14.141227 20195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:33053 every 8 connection(s)
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042033, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:14.161998 19783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:14.145378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:42351 (local address 127.1.245.126:43201)
0806 18:47:14.145635 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:14.145639 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:14.147371 (+ 1732us) server_negotiation.cc:407] Connection header received
0806 18:47:14.148561 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:14.148564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:14.148618 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:14.148662 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:14.150236 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:14.151088 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:14.152733 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:14.153069 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:14.153139 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:14.156178 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:14.156182 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:14.156185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:14.156220 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:14.159583 (+ 3363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:14.160214 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:14.160217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:14.160220 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:14.160309 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:14.160692 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:14.160696 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:14.160698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:14.160927 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:14.160994 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:14.161686 (+ 692us) server_negotiation.cc:299] Negotiation successful
0806 18:47:14.161830 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260806 18:47:14.163753 20199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:14.144061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:43201 (local address 127.1.245.67:42351)
0806 18:47:14.147207 (+ 3146us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:14.147277 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:14.148328 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:14.149073 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:14.149081 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:14.149437 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:14.150081 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:14.150099 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:14.151243 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:14.151247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:14.152601 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:14.152612 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:14.153342 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:14.153346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:14.153547 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:14.154236 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:14.154260 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:14.155986 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:14.159720 (+ 3734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:14.159726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:14.159745 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:14.160102 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:14.160412 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:14.160415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:14.160417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:14.160596 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:14.161073 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:14.161079 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:14.161409 (+ 330us) client_negotiation.cc:769] Sending connection context
0806 18:47:14.162221 (+ 812us) client_negotiation.cc:240] Negotiation successful
0806 18:47:14.162595 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2919,"thread_start_us":158,"threads_started":1}
I20260806 18:47:14.165282 20196 heartbeater.cc:347] Connected to a master server at 127.1.245.126:43201
I20260806 18:47:14.165545 20196 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:14.166150 20196 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:14.167331 19453 ts_manager.cc:194] Registered new tserver with Master: a9ae00555cce4898818ee52bb52a1932 (127.1.245.67:33053)
I20260806 18:47:14.168015 19453 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:42351
I20260806 18:47:14.172170 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:47:14.172983 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 19574
I20260806 18:47:14.224378 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 19789
I20260806 18:47:14.252104 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 19997
I20260806 18:47:14.265146 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 19411
I20260806 18:47:14.276197 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:43201
--webserver_interface=127.1.245.126
--webserver_port=35151
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:43201
--ranger_config_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:55741/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.245.84:55741/kms with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:14.388494 20200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:14.388770 20200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:14.388810 20200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:14.392910 20200 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:14.392980 20200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:14.392998 20200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:14.393013 20200 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:14.393026 20200 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:14.398597 20200 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.245.84:55741/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:43201
--ranger_config_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:43201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=35151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:14.399667 20200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:14.400699 20200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:47:14.409181 20200 server_base.cc:1034] running on GCE node
W20260806 18:47:14.409204 20218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:14.409221 20211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:14.409216 20212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:14.410003 20200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:14.411113 20200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:14.412303 20200 hybrid_clock.cc:648] HybridClock initialized: now 1786042034412274 us; error 37 us; skew 500 ppm
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042034, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:14.415107 20200 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:47:14.416010 20200 webserver.cc:460] Webserver started at http://127.1.245.126:35151/ using document root <none> and password file <none>
I20260806 18:47:14.416544 20200 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:14.416630 20200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042034, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:14.457231 20200 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:47:14.459757 20227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:14.461244 20200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:47:14.461388 20200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "6bd7538e4c28470083b49f5c07d89759"
format_stamp: "Formatted at 2026-08-06 18:47:12 on dist-test-slave-fgb3"
server_key: "2d9f81ccd884f7a9717cb6df304a1ff8"
server_key_iv: "23e97c2a139eb31fe692524bf2d76371"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:14.462030 20200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:14.484056 20200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:14.486872 20200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:14.487058 20200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:14.494375 20200 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:43201
I20260806 18:47:14.494428 20312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:43201 every 8 connection(s)
I20260806 18:47:14.495487 20200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:47:14.499984 20313 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:47:14.501708 20313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Bootstrap starting.
I20260806 18:47:14.503696 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 20200
I20260806 18:47:14.504158 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:55741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:36615
--local_ip_for_outbound_sockets=127.1.245.65
--tserver_master_addrs=127.1.245.126:43201
--webserver_port=38387
--webserver_interface=127.1.245.65
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.245.84:55741/kms with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:47:14.518680 20313 log.cc:824] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:14.525758 20313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:47:14.526197 20313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Bootstrap complete.
I20260806 18:47:14.530437 20313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } }
I20260806 18:47:14.530891 20313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd7538e4c28470083b49f5c07d89759, State: Initialized, Role: FOLLOWER
I20260806 18:47:14.531534 20313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } }
I20260806 18:47:14.531677 20313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:47:14.531756 20313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:47:14.531860 20313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:47:14.533871 20313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } }
I20260806 18:47:14.534246 20313 leader_election.cc:302] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bd7538e4c28470083b49f5c07d89759; no voters:
I20260806 18:47:14.535957 20313 leader_election.cc:288] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:47:14.536198 20319 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:47:14.536490 20319 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [term 2 LEADER]: Becoming Leader. State: Replica: 6bd7538e4c28470083b49f5c07d89759, State: Running, Role: LEADER
I20260806 18:47:14.536872 20319 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } }
I20260806 18:47:14.537467 20313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:14.537940 20321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6bd7538e4c28470083b49f5c07d89759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } } }
I20260806 18:47:14.538080 20321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:14.538762 20322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bd7538e4c28470083b49f5c07d89759. Latest consensus state: current_term: 2 leader_uuid: "6bd7538e4c28470083b49f5c07d89759" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bd7538e4c28470083b49f5c07d89759" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 43201 } } }
I20260806 18:47:14.538856 20322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:14.539693 20324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:47:14.540081 20313 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 18:47:14.546082 20324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:47:14.547005 20324 catalog_manager.cc:1259] Loaded cluster ID: ed29969d93db487da33f03ab6b817ae1
I20260806 18:47:14.547066 20324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:47:14.549706 20324 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:47:14.550446 20324 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Loaded TSK: 0
I20260806 18:47:14.551692 20324 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:47:14.649037 20316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:14.649395 20316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:14.649485 20316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:14.654923 20316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:14.655016 20316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:14.655117 20316 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:47:14.661158 20316 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.245.84:55741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:36615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=38387
--tserver_master_addrs=127.1.245.126:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:14.662303 20316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:14.663453 20316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:14.673149 20380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:14.673312 20378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:14.673708 20316 server_base.cc:1034] running on GCE node
W20260806 18:47:14.673832 20374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:14.674291 20316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:14.675092 20316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:14.676331 20316 hybrid_clock.cc:648] HybridClock initialized: now 1786042034676298 us; error 48 us; skew 500 ppm
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042034, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:14.679060 20316 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:47:14.680012 20316 webserver.cc:460] Webserver started at http://127.1.245.65:38387/ using document root <none> and password file <none>
I20260806 18:47:14.680524 20316 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:14.680583 20316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042034, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:14.713660 20316 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:47:14.717161 20393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:14.718983 20316 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260806 18:47:14.719129 20316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "535ca53409da4be198f8edafa8df7634"
format_stamp: "Formatted at 2026-08-06 18:47:13 on dist-test-slave-fgb3"
server_key: "6e8571341d2af47b2857867c5d9e753e"
server_key_iv: "b8d4661af7d6ff6d9d2e297e5b990b5d"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:14.719909 20316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:14.743311 20316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:14.748719 20316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:14.749027 20316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:14.749902 20316 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:14.751538 20316 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:14.751607 20316 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:14.751667 20316 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:14.751698 20316 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:14.764308 20316 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:36615
I20260806 18:47:14.765900 20316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:47:14.771992 20562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:36615 every 8 connection(s)
I20260806 18:47:14.773655 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 20316
I20260806 18:47:14.774155 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:55741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:33979
--local_ip_for_outbound_sockets=127.1.245.66
--tserver_master_addrs=127.1.245.126:43201
--webserver_port=42387
--webserver_interface=127.1.245.66
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.245.84:55741/kms with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042034, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:14.801321 20568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:14.778495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:43201 (local address 127.1.245.65:53295)
0806 18:47:14.779165 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:14.779227 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:14.780297 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:14.782103 (+ 1806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:14.782112 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:14.782500 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:14.783193 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:14.783212 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:14.784941 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:14.784944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:14.786340 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:14.786353 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:14.787679 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:14.787684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:14.787880 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:14.788895 (+ 1015us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:14.788919 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:14.794737 (+ 5818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:14.797449 (+ 2712us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:14.797456 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:14.797474 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:14.797837 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:14.798297 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:14.798301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:14.798303 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:14.798501 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:14.799320 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:14.799327 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:14.799650 (+ 323us) client_negotiation.cc:769] Sending connection context
0806 18:47:14.799843 (+ 193us) client_negotiation.cc:240] Negotiation successful
0806 18:47:14.800198 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":192,"threads_started":1}
I20260806 18:47:14.803088 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:14.780105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:53295 (local address 127.1.245.126:43201)
0806 18:47:14.780827 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:14.780850 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:14.780896 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:47:14.781177 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:14.781191 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:14.781610 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:14.783441 (+ 1831us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:14.783742 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:14.784788 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:14.786488 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:14.787265 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:14.787434 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:14.795051 (+ 7617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:14.795060 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:14.795081 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:14.795121 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:14.797283 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:14.798007 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:14.798014 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:14.798019 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:14.798180 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:14.798605 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:14.798608 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:14.798610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:14.799124 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:14.799263 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:14.801892 (+ 2629us) server_negotiation.cc:299] Negotiation successful
0806 18:47:14.802193 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":125,"threads_started":1}
I20260806 18:47:14.805804 20563 heartbeater.cc:347] Connected to a master server at 127.1.245.126:43201
I20260806 18:47:14.806048 20563 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:14.806685 20563 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
W20260806 18:47:14.808938 20563 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:43201 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:47:14.809067 20563 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:14.809255 20563 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:14.810063 20563 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:14.810236 20563 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
W20260806 18:47:14.811020 20563 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:47:14.924582 20567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:14.924947 20567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:14.925016 20567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:14.929318 20567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:14.929381 20567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:14.929461 20567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:47:14.935439 20567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.245.84:55741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:33979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=42387
--tserver_master_addrs=127.1.245.126:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:14.936753 20567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:14.937906 20567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:14.949093 20578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:14.949246 20585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:14.949479 20580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:14.949630 20567 server_base.cc:1034] running on GCE node
I20260806 18:47:14.950181 20567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:14.950985 20567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:14.952198 20567 hybrid_clock.cc:648] HybridClock initialized: now 1786042034952158 us; error 45 us; skew 500 ppm
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042034, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:14.956106 20567 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:47:14.957388 20567 webserver.cc:460] Webserver started at http://127.1.245.66:42387/ using document root <none> and password file <none>
I20260806 18:47:14.958189 20567 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:14.958336 20567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:47:14 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042034, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:14.984519 20567 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.003s sys 0.000s
I20260806 18:47:14.993405 20595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:14.995386 20567 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.004s sys 0.000s
I20260806 18:47:14.995546 20567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "7fc8afe4b5c143b293749920c0eb49bf"
format_stamp: "Formatted at 2026-08-06 18:47:13 on dist-test-slave-fgb3"
server_key: "a7a4705a52785d3d41dd10cf18cd5386"
server_key_iv: "11590733c3983cd7660cf9c1c543e36c"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:14.996325 20567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:15.017261 20567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:15.021384 20567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:15.021672 20567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:15.022533 20567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:15.023748 20567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:15.023859 20567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:15.023965 20567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:15.024036 20567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:15.036240 20567 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:33979
I20260806 18:47:15.036346 20784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:33979 every 8 connection(s)
I20260806 18:47:15.037760 20567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:47:15.039206 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 20567
I20260806 18:47:15.039700 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.245.84:55741/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:33053
--local_ip_for_outbound_sockets=127.1.245.67
--tserver_master_addrs=127.1.245.126:43201
--webserver_port=37093
--webserver_interface=127.1.245.67
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.245.84:55741/kms with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:47:15 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042034, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:15.059293 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:15.042607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:42183 (local address 127.1.245.126:43201)
0806 18:47:15.042737 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:15.042741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:15.043031 (+ 290us) server_negotiation.cc:407] Connection header received
0806 18:47:15.044386 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:15.044391 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:15.044454 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:15.044514 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:15.048043 (+ 3529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.048859 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.051060 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.051433 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.051522 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:15.054807 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:15.054812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:15.054815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:15.054849 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:15.056591 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:15.057248 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:15.057253 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:15.057255 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:15.057350 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:15.057789 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:15.057793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:15.057794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:15.058060 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:15.058150 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:15.058966 (+ 816us) server_negotiation.cc:299] Negotiation successful
0806 18:47:15.059110 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:47:15.061367 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:15.041367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:43201 (local address 127.1.245.66:42183)
0806 18:47:15.042843 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:15.042908 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:15.044101 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:15.045004 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:15.045013 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:15.045399 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:15.046129 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:15.046152 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.049037 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.049041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:15.050908 (+ 1867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:15.050920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.051772 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.051776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:15.051978 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:15.052721 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:15.052745 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:15.054575 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:15.056738 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:15.056745 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:15.056765 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:15.057119 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:15.057461 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:15.057464 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:15.057467 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:15.057658 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:15.058226 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:15.058234 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:15.058641 (+ 407us) client_negotiation.cc:769] Sending connection context
0806 18:47:15.059825 (+ 1184us) client_negotiation.cc:240] Negotiation successful
0806 18:47:15.060234 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"thread_start_us":143,"threads_started":1}
I20260806 18:47:15.063097 20785 heartbeater.cc:347] Connected to a master server at 127.1.245.126:43201
I20260806 18:47:15.063366 20785 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:15.064000 20785 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
W20260806 18:47:15.065922 20785 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:43201 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:47:15.066048 20785 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:15.066226 20785 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:15.067157 20785 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:15.067320 20785 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
W20260806 18:47:15.068082 20785 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:47:15.192942 20787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:15.193249 20787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:15.193312 20787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:15.198975 20787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:15.199049 20787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:15.199132 20787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:47:15.205638 20787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.245.84:55741/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:33053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=37093
--tserver_master_addrs=127.1.245.126:43201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:15.206998 20787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:15.208076 20787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:15.218961 20796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:15.219024 20800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:15.219547 20787 server_base.cc:1034] running on GCE node
W20260806 18:47:15.219628 20798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:15.220049 20787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:15.220768 20787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:15.221974 20787 hybrid_clock.cc:648] HybridClock initialized: now 1786042035221942 us; error 41 us; skew 500 ppm
Aug 06 18:47:15 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042035, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:15.225739 20787 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:47:15.227046 20787 webserver.cc:460] Webserver started at http://127.1.245.67:37093/ using document root <none> and password file <none>
I20260806 18:47:15.227842 20787 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:15.227985 20787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 18:47:15 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042035, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:15.253230 20787 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:47:15.256479 20813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:15.258134 20787 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 18:47:15.258280 20787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "a9ae00555cce4898818ee52bb52a1932"
format_stamp: "Formatted at 2026-08-06 18:47:14 on dist-test-slave-fgb3"
server_key: "85279f689540cf231ecece8c593a417d"
server_key_iv: "11f35ed0602575752c891d5564626cd4"
server_key_version: "kuduclusterkey@0"
I20260806 18:47:15.259043 20787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:15.275568 20787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:15.279244 20787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:15.279516 20787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:15.280287 20787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:15.281381 20787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:15.281487 20787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:15.281584 20787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:15.281646 20787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:15.293982 20787 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:33053
I20260806 18:47:15.295454 20787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:47:15.296873 20990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:33053 every 8 connection(s)
I20260806 18:47:15.301831 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 20787
Aug 06 18:47:15 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042035, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:15.319871 20569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:15.297830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:60437 (local address 127.1.245.126:43201)
0806 18:47:15.297968 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:15.297971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:15.299262 (+ 1291us) server_negotiation.cc:407] Connection header received
0806 18:47:15.300619 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:15.300623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:15.300714 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:15.300813 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:15.302674 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.303497 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.305029 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.305415 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.305517 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:15.313072 (+ 7555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:15.313077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:15.313080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:15.313112 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:15.317167 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:15.317882 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:15.317886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:15.317888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:15.317982 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:15.318456 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:15.318460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:15.318462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:15.318691 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:15.318792 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:15.319546 (+ 754us) server_negotiation.cc:299] Negotiation successful
0806 18:47:15.319682 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:47:15.321882 20995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:15.298502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:43201 (local address 127.1.245.67:60437)
0806 18:47:15.299072 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:15.299134 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:15.300303 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:15.301149 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:15.301157 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:15.301518 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:15.302473 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:15.302496 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.303659 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.303663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:15.304892 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:15.304903 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.309136 (+ 4233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.309141 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:15.309389 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:15.311048 (+ 1659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:15.311071 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:15.312844 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:15.317327 (+ 4483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:15.317336 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:15.317360 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:15.317768 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:15.318100 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:15.318103 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:15.318105 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:15.318340 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:15.318865 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:15.318873 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:15.319233 (+ 360us) client_negotiation.cc:769] Sending connection context
0806 18:47:15.320366 (+ 1133us) client_negotiation.cc:240] Negotiation successful
0806 18:47:15.320757 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":148,"threads_started":1}
I20260806 18:47:15.324126 20992 heartbeater.cc:347] Connected to a master server at 127.1.245.126:43201
I20260806 18:47:15.324345 20992 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:15.325017 20992 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:15.326822 21000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:15.307881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46678 (local address 127.1.245.126:43201)
0806 18:47:15.309681 (+ 1800us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:15.309685 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:15.309699 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:47:15.309737 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:15.309739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:15.309784 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:15.310913 (+ 1129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:15.311061 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.312603 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.313826 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:15.314183 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:15.314295 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:15.317741 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:15.317747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:15.317750 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:15.317775 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:15.324162 (+ 6387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:15.324848 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:15.324851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:15.324853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:15.324938 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:15.325355 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:15.325358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:15.325360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:15.325586 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:15.326448 (+ 862us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:15.326524 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 18:47:15.326649 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1717,"thread_start_us":1128,"threads_started":1}
W20260806 18:47:15.326910 20992 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:43201 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 18:47:15.327028 20992 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:15.327190 20992 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:15.328203 20992 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:15.328380 20992 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
W20260806 18:47:15.329150 20992 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 18:47:15 dist-test-slave-fgb3 krb5kdc[18747](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786042035, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:15.811272 20563 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:15.811590 20563 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:15.812826 20249 ts_manager.cc:194] Registered new tserver with Master: 535ca53409da4be198f8edafa8df7634 (127.1.245.65:36615)
I20260806 18:47:15.814713 20249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:53295
Aug 06 18:47:16 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042035, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:16.068323 20785 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:16.068656 20785 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:16.069492 20249 ts_manager.cc:194] Registered new tserver with Master: 7fc8afe4b5c143b293749920c0eb49bf (127.1.245.66:33979)
I20260806 18:47:16.070282 20249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:42183
I20260806 18:47:16.329386 20992 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:16.329703 20992 heartbeater.cc:511] Master 127.1.245.126:43201 requested a full tablet report, sending...
I20260806 18:47:16.330539 20249 ts_manager.cc:194] Registered new tserver with Master: a9ae00555cce4898818ee52bb52a1932 (127.1.245.67:33053)
I20260806 18:47:16.331331 20249 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:60437
I20260806 18:47:16.335377 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 18:47:16 dist-test-slave-fgb3 krb5kdc[18747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042035, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for HTTP/127.1.245.84@KRBTEST.COM
I20260806 18:47:16.357652 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.342771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46692 (local address 127.1.245.126:43201)
0806 18:47:16.346363 (+ 3592us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:16.346368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:16.346385 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:47:16.346432 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:16.346435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:16.346486 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:16.347483 (+ 997us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:16.347595 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.348468 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.349668 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.350039 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.350133 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.351863 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:16.351867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:16.351871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:16.351899 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:16.355423 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:16.355928 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:16.355931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:16.355933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:16.356018 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:16.356342 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:16.356345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:16.356347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:16.356588 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:16.357272 (+ 684us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:16.357344 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 18:47:16.357470 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3480,"thread_start_us":67,"threads_started":1}
I20260806 18:47:16.362388 20249 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:47:16.367561 20249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:47:16.401039 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.390941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:33979 (local address 127.0.0.1:35004)
0806 18:47:16.393696 (+ 2755us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:16.393711 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:16.393810 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:16.395629 (+ 1819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:16.395632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:16.395651 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:16.395743 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.395749 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.397091 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.397094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.399135 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.399145 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.400455 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.400459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.400575 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.400617 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 18:47:16.400741 (+ 124us) client_negotiation.cc:240] Negotiation successful
0806 18:47:16.400876 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2690,"thread_start_us":98,"threads_started":1}
I20260806 18:47:16.402595 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.391373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35004 (local address 127.1.245.66:33979)
0806 18:47:16.394227 (+ 2854us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:16.394233 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:16.394256 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:47:16.394348 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:16.394353 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:16.395346 (+ 993us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:16.395485 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:16.395866 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.396959 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.399275 (+ 2316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.400120 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.400223 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.400387 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:16.401854 (+ 1467us) server_negotiation.cc:299] Negotiation successful
0806 18:47:16.402444 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2751,"thread_start_us":99,"threads_started":1}
I20260806 18:47:16.406169 20669 tablet_service.cc:1467] Processing CreateTablet for tablet f7d6111ec1564ee5922ebeae2ebf7a3d (DEFAULT_TABLE table=test-table [id=7d120550f1334489bf1f8c0bcd969796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:16.407433 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.388019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:33053 (local address 127.0.0.1:34910)
0806 18:47:16.391017 (+ 2998us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:16.391072 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:16.391210 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:16.394057 (+ 2847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:16.394064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:16.394088 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:16.395197 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.395208 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.402829 (+ 7621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.402834 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.404893 (+ 2059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.404905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.407021 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.407025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.407120 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.407144 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:47:16.407208 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:47:16.407299 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2817,"thread_start_us":95,"threads_started":1}
I20260806 18:47:16.407923 20669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d6111ec1564ee5922ebeae2ebf7a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:16.408994 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.389296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34910 (local address 127.1.245.67:33053)
0806 18:47:16.389795 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:16.389802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:16.392338 (+ 2536us) server_negotiation.cc:407] Connection header received
0806 18:47:16.392523 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:16.392529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:16.392759 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:16.393868 (+ 1109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:16.395345 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.401712 (+ 6367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.405050 (+ 3338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.406718 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.406820 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.406961 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:16.407714 (+ 753us) server_negotiation.cc:299] Negotiation successful
0806 18:47:16.408856 (+ 1142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":120,"threads_started":1}
I20260806 18:47:16.412061 20886 tablet_service.cc:1467] Processing CreateTablet for tablet f7d6111ec1564ee5922ebeae2ebf7a3d (DEFAULT_TABLE table=test-table [id=7d120550f1334489bf1f8c0bcd969796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:16.413182 20886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d6111ec1564ee5922ebeae2ebf7a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:16.418963 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.392880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:36615 (local address 127.0.0.1:47502)
0806 18:47:16.398458 (+ 5578us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:16.398473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:16.398571 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:16.399214 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:16.399217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:16.399235 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:16.399333 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.399339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.400788 (+ 1449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.400792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.402299 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.402344 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.403614 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.403618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.418574 (+ 14956us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.418615 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:47:16.418706 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 18:47:16.418811 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5516,"thread_start_us":85,"threads_started":1}
I20260806 18:47:16.419485 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.393268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47502 (local address 127.1.245.65:36615)
0806 18:47:16.394038 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:16.394046 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:16.398566 (+ 4520us) server_negotiation.cc:407] Connection header received
0806 18:47:16.398680 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:16.398686 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:16.398891 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:16.399059 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:16.399470 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.400658 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.402488 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.403300 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.403400 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.403550 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:16.419190 (+ 15640us) server_negotiation.cc:299] Negotiation successful
0806 18:47:16.419339 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":124,"threads_started":1}
I20260806 18:47:16.422535 20447 tablet_service.cc:1467] Processing CreateTablet for tablet f7d6111ec1564ee5922ebeae2ebf7a3d (DEFAULT_TABLE table=test-table [id=7d120550f1334489bf1f8c0bcd969796]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:16.423502 20447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d6111ec1564ee5922ebeae2ebf7a3d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:16.425841 21050 tablet_bootstrap.cc:492] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: Bootstrap starting.
I20260806 18:47:16.428635 21050 tablet_bootstrap.cc:654] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:16.429661 21050 log.cc:824] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:16.432178 21050 tablet_bootstrap.cc:492] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: No bootstrap required, opened a new log
I20260806 18:47:16.432425 21050 ts_tablet_manager.cc:1397] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:47:16.437268 21050 raft_consensus.cc:348] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.437745 21050 raft_consensus.cc:374] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:16.437866 21050 raft_consensus.cc:729] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fc8afe4b5c143b293749920c0eb49bf, State: Initialized, Role: FOLLOWER
I20260806 18:47:16.438587 21050 consensus_queue.cc:260] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.439715 20785 heartbeater.cc:503] Master 127.1.245.126:43201 was elected leader, sending a full tablet report...
I20260806 18:47:16.439860 21050 ts_tablet_manager.cc:1428] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:47:16.440430 21052 tablet_bootstrap.cc:492] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634: Bootstrap starting.
I20260806 18:47:16.443131 21052 tablet_bootstrap.cc:654] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:16.443642 21053 tablet_bootstrap.cc:492] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Bootstrap starting.
I20260806 18:47:16.446607 21053 tablet_bootstrap.cc:654] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:16.447562 21053 log.cc:824] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:16.448496 21052 log.cc:824] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:16.449507 21060 raft_consensus.cc:482] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:16.449748 21060 raft_consensus.cc:504] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.453982 21060 leader_election.cc:288] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9ae00555cce4898818ee52bb52a1932 (127.1.245.67:33053), 535ca53409da4be198f8edafa8df7634 (127.1.245.65:36615)
I20260806 18:47:16.456220 21052 tablet_bootstrap.cc:492] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634: No bootstrap required, opened a new log
I20260806 18:47:16.456537 21052 ts_tablet_manager.cc:1397] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634: Time spent bootstrapping tablet: real 0.016s user 0.006s sys 0.000s
I20260806 18:47:16.461057 21052 raft_consensus.cc:348] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.461773 21052 raft_consensus.cc:374] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:16.461887 21052 raft_consensus.cc:729] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 535ca53409da4be198f8edafa8df7634, State: Initialized, Role: FOLLOWER
I20260806 18:47:16.462662 21052 consensus_queue.cc:260] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.466751 21052 ts_tablet_manager.cc:1428] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260806 18:47:16.468695 20563 heartbeater.cc:503] Master 127.1.245.126:43201 was elected leader, sending a full tablet report...
I20260806 18:47:16.471870 21044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.453040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:55673 (local address 127.1.245.67:33053)
0806 18:47:16.453165 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:16.453170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:16.461701 (+ 8531us) server_negotiation.cc:407] Connection header received
0806 18:47:16.462985 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:16.462990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:16.463047 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:16.463100 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:16.463572 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.464380 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.467627 (+ 3247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.468356 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.468459 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.468515 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:16.471650 (+ 3135us) server_negotiation.cc:299] Negotiation successful
0806 18:47:16.471744 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:47:16.472506 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.452647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:33053 (local address 127.1.245.66:55673)
0806 18:47:16.461593 (+ 8946us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:16.461610 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:16.462069 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:16.463258 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:16.463262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:16.463282 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:16.463432 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.463439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.464545 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.464548 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.467251 (+ 2703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.467263 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.471241 (+ 3978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.471245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.471360 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.471389 (+ 29us) client_negotiation.cc:769] Sending connection context
0806 18:47:16.472238 (+ 849us) client_negotiation.cc:240] Negotiation successful
0806 18:47:16.472330 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8861,"thread_start_us":90,"threads_started":1}
I20260806 18:47:16.473651 20919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d6111ec1564ee5922ebeae2ebf7a3d" candidate_uuid: "7fc8afe4b5c143b293749920c0eb49bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ae00555cce4898818ee52bb52a1932" is_pre_election: true
W20260806 18:47:16.475864 20600 leader_election.cc:341] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9ae00555cce4898818ee52bb52a1932 (127.1.245.67:33053): Illegal state: must be running to vote when last-logged opid is not known
I20260806 18:47:16.477165 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.453821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:48421 (local address 127.1.245.65:36615)
0806 18:47:16.453948 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:16.453952 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:16.473285 (+ 19333us) server_negotiation.cc:407] Connection header received
0806 18:47:16.473469 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:16.473472 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:16.473521 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:16.473567 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:16.473925 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.474689 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.475992 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.476513 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.476604 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.476656 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:16.476982 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 18:47:16.477055 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:47:16.477523 21066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.453480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:36615 (local address 127.1.245.66:48421)
0806 18:47:16.473216 (+ 19736us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:16.473230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:16.473351 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:16.473704 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:16.473706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:16.473723 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:16.473799 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.473804 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.474827 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.474830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.475845 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:16.475855 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.476751 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.476754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:16.476834 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.476857 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:47:16.477335 (+ 478us) client_negotiation.cc:240] Negotiation successful
0806 18:47:16.477404 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19675,"thread_start_us":100,"threads_started":1}
I20260806 18:47:16.478101 20494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d6111ec1564ee5922ebeae2ebf7a3d" candidate_uuid: "7fc8afe4b5c143b293749920c0eb49bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "535ca53409da4be198f8edafa8df7634" is_pre_election: true
I20260806 18:47:16.478400 20494 raft_consensus.cc:2454] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7fc8afe4b5c143b293749920c0eb49bf in term 0.
I20260806 18:47:16.478624 21053 tablet_bootstrap.cc:492] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: No bootstrap required, opened a new log
I20260806 18:47:16.478863 21053 ts_tablet_manager.cc:1397] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Time spent bootstrapping tablet: real 0.035s user 0.006s sys 0.000s
I20260806 18:47:16.478820 20596 leader_election.cc:302] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 535ca53409da4be198f8edafa8df7634, 7fc8afe4b5c143b293749920c0eb49bf; no voters: a9ae00555cce4898818ee52bb52a1932
I20260806 18:47:16.481247 21053 raft_consensus.cc:348] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.481442 21053 raft_consensus.cc:374] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:16.481473 21053 raft_consensus.cc:729] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9ae00555cce4898818ee52bb52a1932, State: Initialized, Role: FOLLOWER
I20260806 18:47:16.481863 21053 consensus_queue.cc:260] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.482518 20992 heartbeater.cc:503] Master 127.1.245.126:43201 was elected leader, sending a full tablet report...
I20260806 18:47:16.482573 21058 raft_consensus.cc:2781] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:47:16.482664 21058 raft_consensus.cc:482] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:47:16.482705 21058 raft_consensus.cc:3037] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:16.484011 21058 raft_consensus.cc:504] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.484730 20919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d6111ec1564ee5922ebeae2ebf7a3d" candidate_uuid: "7fc8afe4b5c143b293749920c0eb49bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9ae00555cce4898818ee52bb52a1932"
I20260806 18:47:16.484876 20919 raft_consensus.cc:3037] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:16.485888 20919 raft_consensus.cc:2454] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fc8afe4b5c143b293749920c0eb49bf in term 1.
I20260806 18:47:16.486199 20600 leader_election.cc:302] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7fc8afe4b5c143b293749920c0eb49bf, a9ae00555cce4898818ee52bb52a1932; no voters:
I20260806 18:47:16.486428 21060 raft_consensus.cc:2781] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:16.486681 21060 raft_consensus.cc:686] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 1 LEADER]: Becoming Leader. State: Replica: 7fc8afe4b5c143b293749920c0eb49bf, State: Running, Role: LEADER
I20260806 18:47:16.487017 21060 consensus_queue.cc:237] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } }
I20260806 18:47:16.487227 21058 leader_election.cc:288] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [CANDIDATE]: Term 1 election: Requested vote from peers a9ae00555cce4898818ee52bb52a1932 (127.1.245.67:33053), 535ca53409da4be198f8edafa8df7634 (127.1.245.65:36615)
I20260806 18:47:16.487308 21053 ts_tablet_manager.cc:1428] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260806 18:47:16.488919 20494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d6111ec1564ee5922ebeae2ebf7a3d" candidate_uuid: "7fc8afe4b5c143b293749920c0eb49bf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "535ca53409da4be198f8edafa8df7634"
I20260806 18:47:16.489056 20494 raft_consensus.cc:3037] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:16.490432 20494 raft_consensus.cc:2454] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7fc8afe4b5c143b293749920c0eb49bf in term 1.
I20260806 18:47:16.491173 20251 catalog_manager.cc:5495] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf reported cstate change: term changed from 0 to 1, leader changed from <none> to 7fc8afe4b5c143b293749920c0eb49bf (127.1.245.66). New cstate: current_term: 1 leader_uuid: "7fc8afe4b5c143b293749920c0eb49bf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7fc8afe4b5c143b293749920c0eb49bf" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 } health_report { overall_health: UNKNOWN } } }
I20260806 18:47:16.526305 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.521058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35008 (local address 127.1.245.66:33979)
0806 18:47:16.521185 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:16.521189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:16.521203 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:47:16.521297 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:16.521301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:16.521382 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:16.521490 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:47:16.521889 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.522719 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.523994 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:16.524357 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:16.524453 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:16.524927 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:47:16.525489 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:47:16.525606 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:16.526090 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 18:47:16.526155 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260806 18:47:16.527742 20565 tablet.cc:2369] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:16.534098 20494 raft_consensus.cc:1261] T f7d6111ec1564ee5922ebeae2ebf7a3d P 535ca53409da4be198f8edafa8df7634 [term 1 FOLLOWER]: Refusing update from remote peer 7fc8afe4b5c143b293749920c0eb49bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:16.535063 20919 raft_consensus.cc:1261] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [term 1 FOLLOWER]: Refusing update from remote peer 7fc8afe4b5c143b293749920c0eb49bf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:16.535545 21060 consensus_queue.cc:1035] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "535ca53409da4be198f8edafa8df7634" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 36615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:16.535745 21058 consensus_queue.cc:1035] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9ae00555cce4898818ee52bb52a1932" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 33053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:47:16.547813 20994 tablet.cc:2369] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:16.553764 21074 mvcc.cc:205] Tried to move back new op lower bound from 7315628181634400256 to 7315628181454028800. Current Snapshot: MvccSnapshot[applied={T|T < 7315628181634400256}]
I20260806 18:47:16.554169 21078 mvcc.cc:205] Tried to move back new op lower bound from 7315628181634400256 to 7315628181454028800. Current Snapshot: MvccSnapshot[applied={T|T < 7315628181634400256}]
I20260806 18:47:16.566767 21075 mvcc.cc:205] Tried to move back new op lower bound from 7315628181634400256 to 7315628181454028800. Current Snapshot: MvccSnapshot[applied={T|T < 7315628181634400256 or (T in {7315628181634400256})}]
I20260806 18:47:16.566804 20251 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:47:16.566993 20251 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:47:16.570109 20251 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6bd7538e4c28470083b49f5c07d89759: Sending DeleteTablet for 3 replicas of tablet f7d6111ec1564ee5922ebeae2ebf7a3d
I20260806 18:47:16.571312 20447 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d6111ec1564ee5922ebeae2ebf7a3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:16 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:47502
I20260806 18:47:16.572052 20669 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d6111ec1564ee5922ebeae2ebf7a3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:16 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:35004
I20260806 18:47:16.572530 20886 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d6111ec1564ee5922ebeae2ebf7a3d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:16 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:34910
I20260806 18:47:16.572885 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 20316
I20260806 18:47:16.582667 21093 tablet_replica.cc:330] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: stopping tablet replica
I20260806 18:47:16.583197 21093 raft_consensus.cc:2229] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:47:16.583240 21091 tablet_replica.cc:330] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: stopping tablet replica
I20260806 18:47:16.583514 21091 raft_consensus.cc:2229] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:47:16.583806 21093 raft_consensus.cc:2258] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:16.583904 21091 raft_consensus.cc:2258] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:16.585510 21091 ts_tablet_manager.cc:1905] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:16.585510 21093 ts_tablet_manager.cc:1905] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:16.588411 21091 ts_tablet_manager.cc:1918] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:16.588411 21093 ts_tablet_manager.cc:1918] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:16.588527 21091 log.cc:1196] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/f7d6111ec1564ee5922ebeae2ebf7a3d
I20260806 18:47:16.588645 21093 log.cc:1196] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/f7d6111ec1564ee5922ebeae2ebf7a3d
I20260806 18:47:16.588882 21091 ts_tablet_manager.cc:1939] T f7d6111ec1564ee5922ebeae2ebf7a3d P 7fc8afe4b5c143b293749920c0eb49bf: Deleting consensus metadata
I20260806 18:47:16.589102 21093 ts_tablet_manager.cc:1939] T f7d6111ec1564ee5922ebeae2ebf7a3d P a9ae00555cce4898818ee52bb52a1932: Deleting consensus metadata
I20260806 18:47:16.590232 20231 catalog_manager.cc:4841] TS a9ae00555cce4898818ee52bb52a1932 (127.1.245.67:33053): tablet f7d6111ec1564ee5922ebeae2ebf7a3d (table test-table [id=7d120550f1334489bf1f8c0bcd969796]) successfully deleted
I20260806 18:47:16.590232 20230 catalog_manager.cc:4841] TS 7fc8afe4b5c143b293749920c0eb49bf (127.1.245.66:33979): tablet f7d6111ec1564ee5922ebeae2ebf7a3d (table test-table [id=7d120550f1334489bf1f8c0bcd969796]) successfully deleted
W20260806 18:47:16.596040 20228 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:36615 (error 108)
I20260806 18:47:16.596266 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 20567
I20260806 18:47:16.607424 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:16.607019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:36615 (local address 127.0.0.1:47518)
0806 18:47:16.607239 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:16.607318 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:36615: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134}
W20260806 18:47:16.608006 20228 catalog_manager.cc:4584] TS 535ca53409da4be198f8edafa8df7634 (127.1.245.65:36615): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f7d6111ec1564ee5922ebeae2ebf7a3d: Network error: Client connection negotiation failed: client connection to 127.1.245.65:36615: connect: Connection refused (error 111)
I20260806 18:47:16.615310 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 20787
I20260806 18:47:16.627014 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 20200
I20260806 18:47:16.645665 2005 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 18:47:16.990895 2005 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 18:47:16.991017 2005 mini_ranger.cc:66] Stopping Ranger...
I20260806 18:47:17.336107 2005 mini_ranger.cc:68] Stopped Ranger
2026-08-06 18:47:17.336 UTC [18794] LOG: received smart shutdown request
2026-08-06 18:47:17.340 UTC [18794] LOG: background worker "logical replication launcher" (PID 18804) exited with exit code 1
2026-08-06 18:47:17.340 UTC [18798] LOG: shutting down
2026-08-06 18:47:17.500 UTC [18794] LOG: database system is shut down
2026-08-06T18:47:17Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48304 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:47:17 dist-test-slave-fgb3 krb5kdc[21109](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:47:17 dist-test-slave-fgb3 krb5kdc[21109](info): set up 2 sockets
Aug 06 18:47:17 dist-test-slave-fgb3 krb5kdc[21109](info): commencing operation
krb5kdc: starting...
W20260806 18:47:18.727128 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:47:18 dist-test-slave-fgb3 krb5kdc[21109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:47:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:47:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:47:18.845499 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35837
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35837,127.1.245.125:41033,127.1.245.124:33375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:18.954991 21128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:18.955217 21128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:18.955253 21128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:18.959162 21128 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:18.959209 21128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:18.959227 21128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:18.959242 21128 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:18.959271 21128 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:18.964300 21128 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35837,127.1.245.125:41033,127.1.245.124:33375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:18.965309 21128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:18.966292 21128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:18.974500 21136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:18.974565 21128 server_base.cc:1034] running on GCE node
W20260806 18:47:18.974725 21137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:18.974733 21139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:18.975340 21128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:18.976387 21128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:18.977553 21128 hybrid_clock.cc:648] HybridClock initialized: now 1786042038977535 us; error 30 us; skew 500 ppm
Aug 06 18:47:18 dist-test-slave-fgb3 krb5kdc[21109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:18.980217 21128 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:47:18.981061 21128 webserver.cc:460] Webserver started at http://127.1.245.126:34687/ using document root <none> and password file <none>
I20260806 18:47:18.981580 21128 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:18.981644 21128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:18.981846 21128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:18.983884 21128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "f966f7c8cec74b69a02bff57e0c50308"
format_stamp: "Formatted at 2026-08-06 18:47:18 on dist-test-slave-fgb3"
server_key: "b2d355b39d148c1db51421d4e9ce3ffa"
server_key_iv: "2a18bd4efe7ec63a0b59c9161b0d9133"
server_key_version: "encryptionkey@0"
I20260806 18:47:18.984417 21128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "f966f7c8cec74b69a02bff57e0c50308"
format_stamp: "Formatted at 2026-08-06 18:47:18 on dist-test-slave-fgb3"
server_key: "b2d355b39d148c1db51421d4e9ce3ffa"
server_key_iv: "2a18bd4efe7ec63a0b59c9161b0d9133"
server_key_version: "encryptionkey@0"
I20260806 18:47:18.987998 21128 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:47:18.990438 21150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:18.991641 21128 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:47:18.991771 21128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "f966f7c8cec74b69a02bff57e0c50308"
format_stamp: "Formatted at 2026-08-06 18:47:18 on dist-test-slave-fgb3"
server_key: "b2d355b39d148c1db51421d4e9ce3ffa"
server_key_iv: "2a18bd4efe7ec63a0b59c9161b0d9133"
server_key_version: "encryptionkey@0"
I20260806 18:47:18.991865 21128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:19.013952 21128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:19.016840 21128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:19.017020 21128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:19.024097 21128 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:35837
I20260806 18:47:19.024122 21228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:35837 every 8 connection(s)
I20260806 18:47:19.025103 21128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:47:19.027988 21230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:19.031683 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 21128
I20260806 18:47:19.031824 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:47:19.032106 2005 external_mini_cluster.cc:1428] Setting key 98f97f99b73ea6379f3e0bfec3e415d0
I20260806 18:47:19.031970 21230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.245.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.125@KRBTEST.COM" created.
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:19.049801 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.034726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35837 (local address 127.0.0.1:49054)
0806 18:47:19.035592 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.035646 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.036705 (+ 1059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.037489 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.037495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.037986 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.038395 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.038417 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.039654 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.039658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.040837 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.040850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.041551 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.041554 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.041651 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.042267 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.042288 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.044932 (+ 2644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.046888 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.046898 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.046911 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.047189 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.047546 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.047548 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.047551 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.047695 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.048339 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.048347 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.048524 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.048707 (+ 183us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.049101 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":66,"thread_start_us":158,"threads_started":1}
I20260806 18:47:19.049805 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.034726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.1.245.126:35837)
0806 18:47:19.035393 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.035411 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.035794 (+ 383us) server_negotiation.cc:407] Connection header received
0806 18:47:19.037075 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.037080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.037281 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.037434 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.038579 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.039524 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.040971 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.041404 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.041525 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.045098 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.045105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.045119 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.045146 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.046777 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.047350 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.047355 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.047356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.047460 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.047784 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.047787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.047788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.048200 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.048365 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.048775 (+ 410us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.048970 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"spinlock_wait_cycles":70912,"thread_start_us":197,"threads_started":1}
I20260806 18:47:19.052266 21230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:47:19.052980 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.052713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:60958)
0806 18:47:19.052847 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.052919 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 18:47:19.053225 21158 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
I20260806 18:47:19.054153 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.053945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:60960)
0806 18:47:19.054051 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.054100 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:47:19.054376 21230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:41033: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab.
Entry for principal kudu/127.1.245.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab.
I20260806 18:47:19.061111 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.125:41033
--webserver_interface=127.1.245.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab
--principal=kudu/127.1.245.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35837,127.1.245.125:41033,127.1.245.124:33375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:47:19.103609 21230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } attempt: 1
I20260806 18:47:19.104504 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.104198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:60970)
0806 18:47:19.104352 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.104418 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 18:47:19.105201 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.105001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:60978)
0806 18:47:19.105094 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.105140 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:47:19.105327 21230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:41033: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
W20260806 18:47:19.172801 21240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:19.173043 21240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:19.173081 21240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:47:19.173481 21230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } attempt: 2
I20260806 18:47:19.174357 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.174013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:60988)
0806 18:47:19.174167 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.174241 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:47:19.175036 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.174862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:32772)
0806 18:47:19.174942 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.174983 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 18:47:19.175151 21230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:41033: Network error: Client connection negotiation failed: client connection to 127.1.245.125:41033: connect: Connection refused (error 111)
W20260806 18:47:19.177433 21240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:19.177484 21240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:19.177502 21240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:19.177531 21240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:19.177561 21240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:19.182950 21240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35837,127.1.245.125:41033,127.1.245.124:33375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab
--principal=kudu/127.1.245.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.125:41033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.245.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:19.183947 21240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:19.184950 21240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:19.193324 21249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:19.193313 21247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:19.193316 21246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:19.193470 21240 server_base.cc:1034] running on GCE node
I20260806 18:47:19.194100 21240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:19.195250 21240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:19.196452 21240 hybrid_clock.cc:648] HybridClock initialized: now 1786042039196432 us; error 27 us; skew 500 ppm
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:19.199314 21240 init.cc:377] Logged in from keytab as kudu/127.1.245.125@KRBTEST.COM (short username kudu)
I20260806 18:47:19.200193 21240 webserver.cc:460] Webserver started at http://127.1.245.125:41281/ using document root <none> and password file <none>
I20260806 18:47:19.200723 21240 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:19.200806 21240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:19.201014 21240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:19.203094 21240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/data/instance:
uuid: "e047dbee558b4c63a59ae4efcc2b42e7"
format_stamp: "Formatted at 2026-08-06 18:47:19 on dist-test-slave-fgb3"
server_key: "3b648d6142e7b9d492efe6e126a5ddeb"
server_key_iv: "922426c1f937c729562a6bcd02bf7216"
server_key_version: "encryptionkey@0"
I20260806 18:47:19.203639 21240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/wal/instance:
uuid: "e047dbee558b4c63a59ae4efcc2b42e7"
format_stamp: "Formatted at 2026-08-06 18:47:19 on dist-test-slave-fgb3"
server_key: "3b648d6142e7b9d492efe6e126a5ddeb"
server_key_iv: "922426c1f937c729562a6bcd02bf7216"
server_key_version: "encryptionkey@0"
I20260806 18:47:19.207334 21240 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:47:19.209760 21265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:19.211033 21240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:47:19.211154 21240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/wal
uuid: "e047dbee558b4c63a59ae4efcc2b42e7"
format_stamp: "Formatted at 2026-08-06 18:47:19 on dist-test-slave-fgb3"
server_key: "3b648d6142e7b9d492efe6e126a5ddeb"
server_key_iv: "922426c1f937c729562a6bcd02bf7216"
server_key_version: "encryptionkey@0"
I20260806 18:47:19.211253 21240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:19.222116 21240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:19.224948 21240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:19.225127 21240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:19.232167 21240 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.125:41033
I20260806 18:47:19.232268 21353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.125:41033 every 8 connection(s)
I20260806 18:47:19.233144 21240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
I20260806 18:47:19.235988 21355 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:19.237869 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 21240
I20260806 18:47:19.237989 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-1/wal/instance
I20260806 18:47:19.238241 2005 external_mini_cluster.cc:1428] Setting key 114ea74b68cd93feb8c5cccb0c8ff7c1
I20260806 18:47:19.240049 21355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.245.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.124@KRBTEST.COM" created.
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:19.254684 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.242425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49062 (local address 127.1.245.126:35837)
0806 18:47:19.242600 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.242605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.243551 (+ 946us) server_negotiation.cc:407] Connection header received
0806 18:47:19.244727 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.244730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.244786 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.244871 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.246444 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.247008 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.248331 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.248580 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.248685 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.251193 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.251196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.251198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.251227 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.252635 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.253135 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.253138 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.253139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.253215 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.253538 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.253541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.253542 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.253720 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.253812 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.254374 (+ 562us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.254504 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:47:19.255473 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.242792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35837 (local address 127.0.0.1:49062)
0806 18:47:19.243355 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.243415 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.244479 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.245071 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.245080 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.245561 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.246257 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.246279 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.247153 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.247157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.248185 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.248197 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.248750 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.248754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.248930 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.249505 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.249522 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.251020 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.252756 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.252762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.252778 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.253027 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.253300 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.253303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.253305 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.253441 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.253829 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.253836 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.254099 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.254269 (+ 170us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.254617 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":137,"threads_started":1}
I20260806 18:47:19.256979 21355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab.
Entry for principal kudu/127.1.245.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab.
I20260806 18:47:19.265465 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.124:33375
--webserver_interface=127.1.245.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab
--principal=kudu/127.1.245.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35837,127.1.245.125:41033,127.1.245.124:33375
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:47:19.265990 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.257512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:32774)
0806 18:47:19.257627 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.257641 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.257714 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.258365 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.258368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.258517 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.258652 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.258661 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.259631 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.259634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.260248 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.260287 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.260941 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.260943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.261020 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.261494 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.261508 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.262720 (+ 1212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.264405 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.264407 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.264409 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.264592 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.264934 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.264938 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.264940 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.265058 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.265494 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.265497 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.265598 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.265704 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.265825 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260806 18:47:19.266057 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.257649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32774 (local address 127.1.245.125:41033)
0806 18:47:19.258009 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.258018 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.258037 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:47:19.258069 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.258073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.258213 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.258363 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.258759 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.259505 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.260416 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.260835 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.260928 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.262856 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.262865 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.262875 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.262899 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.264291 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.264712 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.264718 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.264720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.264815 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.265149 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.265151 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.265152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.265386 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.265507 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.265782 (+ 275us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.265906 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":170,"threads_started":1}
I20260806 18:47:19.266292 21230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } attempt: 3
I20260806 18:47:19.268160 21355 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:47:19.268877 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.268624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35300)
0806 18:47:19.268734 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.268813 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:47:19.269719 21270 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
I20260806 18:47:19.270712 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.270493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35310)
0806 18:47:19.270596 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.270649 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 18:47:19.271236 21355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:33375: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
I20260806 18:47:19.275247 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.266780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32788 (local address 127.1.245.125:41033)
0806 18:47:19.266878 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.266882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.267113 (+ 231us) server_negotiation.cc:407] Connection header received
0806 18:47:19.267149 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.267152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.267190 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.267257 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.268153 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.268815 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.269615 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.270483 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.270578 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.272545 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.272548 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.272550 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.272571 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.273785 (+ 1214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.274172 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.274175 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.274177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.274249 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.274522 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.274525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.274527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.274685 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.274769 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.275018 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.275116 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:47:19.275250 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.266738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:32788)
0806 18:47:19.266918 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.266932 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.267037 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.267312 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.267316 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.267489 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.268050 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.268061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.268919 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.268923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.269500 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.269511 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.270689 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.270692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.270789 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.271253 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.271265 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.272418 (+ 1153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.273887 (+ 1469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.273890 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.273892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.274084 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.274353 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.274356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.274359 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.274439 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.274776 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.274779 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.274871 (+ 92us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.274966 (+ 95us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.275099 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 18:47:19.275949 21230 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:47:19.276589 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.276369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35318)
0806 18:47:19.276475 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.276526 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 18:47:19.277235 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.277014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35328)
0806 18:47:19.277142 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.277185 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260806 18:47:19.277364 21230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:33375: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
I20260806 18:47:19.308557 21230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } attempt: 1
I20260806 18:47:19.309499 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.309159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35342)
0806 18:47:19.309324 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.309387 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260806 18:47:19.310218 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.310003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35350)
0806 18:47:19.310109 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.310155 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:47:19.310392 21230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:33375: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
I20260806 18:47:19.320461 21355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } attempt: 1
I20260806 18:47:19.321419 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.321030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35362)
0806 18:47:19.321261 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.321323 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260806 18:47:19.322256 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.321986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35372)
0806 18:47:19.322133 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.322178 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260806 18:47:19.322480 21355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:33375: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
I20260806 18:47:19.385561 21230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } attempt: 2
I20260806 18:47:19.386471 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.386115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35376)
0806 18:47:19.386276 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.386382 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 18:47:19.387233 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.386974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35380)
0806 18:47:19.387101 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.387157 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:47:19.387398 21230 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:33375: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
W20260806 18:47:19.388723 21367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:19.388968 21367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:19.389027 21367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:47:19.390621 21355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } attempt: 2
I20260806 18:47:19.391330 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.391037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35394)
0806 18:47:19.391166 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.391250 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 18:47:19.391999 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.391786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35410)
0806 18:47:19.391878 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.391926 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 18:47:19.392112 21355 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:33375: Network error: Client connection negotiation failed: client connection to 127.1.245.124:33375: connect: Connection refused (error 111)
W20260806 18:47:19.393481 21367 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:19.393529 21367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:19.393548 21367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:19.393563 21367 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:19.393586 21367 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:19.398969 21367 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35837,127.1.245.125:41033,127.1.245.124:33375
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab
--principal=kudu/127.1.245.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.124:33375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.245.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:19.399998 21367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:19.401000 21367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:19.409590 21378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:19.409590 21377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:19.409607 21380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:19.409680 21367 server_base.cc:1034] running on GCE node
I20260806 18:47:19.410475 21367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:19.411584 21367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:19.412863 21367 hybrid_clock.cc:648] HybridClock initialized: now 1786042039412774 us; error 29 us; skew 500 ppm
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:19.415846 21367 init.cc:377] Logged in from keytab as kudu/127.1.245.124@KRBTEST.COM (short username kudu)
I20260806 18:47:19.416749 21367 webserver.cc:460] Webserver started at http://127.1.245.124:33533/ using document root <none> and password file <none>
I20260806 18:47:19.417320 21367 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:19.417393 21367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:19.417598 21367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:19.419667 21367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/data/instance:
uuid: "980e93640bf44fc5a993088a17c7de20"
format_stamp: "Formatted at 2026-08-06 18:47:19 on dist-test-slave-fgb3"
server_key: "fd883b2fdb853b8a6229980c5b8a000e"
server_key_iv: "b170be04a88fe717ef117fd063efda6b"
server_key_version: "encryptionkey@0"
I20260806 18:47:19.420197 21367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/wal/instance:
uuid: "980e93640bf44fc5a993088a17c7de20"
format_stamp: "Formatted at 2026-08-06 18:47:19 on dist-test-slave-fgb3"
server_key: "fd883b2fdb853b8a6229980c5b8a000e"
server_key_iv: "b170be04a88fe717ef117fd063efda6b"
server_key_version: "encryptionkey@0"
I20260806 18:47:19.423902 21367 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:47:19.426440 21391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:19.427621 21367 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:47:19.427747 21367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/wal
uuid: "980e93640bf44fc5a993088a17c7de20"
format_stamp: "Formatted at 2026-08-06 18:47:19 on dist-test-slave-fgb3"
server_key: "fd883b2fdb853b8a6229980c5b8a000e"
server_key_iv: "b170be04a88fe717ef117fd063efda6b"
server_key_version: "encryptionkey@0"
I20260806 18:47:19.427841 21367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:19.448630 21367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:19.451690 21367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:19.451882 21367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:19.458938 21367 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.124:33375
I20260806 18:47:19.458954 21473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.124:33375 every 8 connection(s)
I20260806 18:47:19.459903 21367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
I20260806 18:47:19.462507 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 21367
I20260806 18:47:19.462630 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786041874296929-2005-0/minicluster-data/master-2/wal/instance
I20260806 18:47:19.462862 21474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:19.462917 2005 external_mini_cluster.cc:1428] Setting key d7a21105f1af11a04803b22671a02a24
I20260806 18:47:19.468075 21474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:19.477571 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.464390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35414 (local address 127.1.245.124:33375)
0806 18:47:19.464855 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.464882 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.464928 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:47:19.465664 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.465689 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.466126 (+ 437us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.466516 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.467186 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.468647 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.469506 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.470139 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.470290 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.472735 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.472742 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.472754 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.472788 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.474605 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.475063 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.475070 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.475075 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.475217 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.475507 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.475510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.475511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.475898 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.476061 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.476385 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.476683 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":114,"threads_started":1}
I20260806 18:47:19.478718 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.469880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35837 (local address 127.0.0.1:49074)
0806 18:47:19.470304 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.470377 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.470528 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.470816 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.470822 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.471155 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.471294 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.471300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.472024 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.472028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.472708 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.472717 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.473176 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.473180 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.473306 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.473715 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.473730 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.475326 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.477071 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.477090 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.477095 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.477369 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.477613 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.477616 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.477618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.477706 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.478116 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.478122 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.478236 (+ 114us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.478403 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.478565 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":152,"threads_started":1}
I20260806 18:47:19.478787 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.469956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49074 (local address 127.1.245.126:35837)
0806 18:47:19.470101 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.470106 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.470445 (+ 339us) server_negotiation.cc:407] Connection header received
0806 18:47:19.470634 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.470637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.470683 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.470752 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.471424 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.471912 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.472828 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.473063 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.473165 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.475461 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.475465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.475468 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.475498 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.476969 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.477459 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.477462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.477463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.477533 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.477795 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.477798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.477800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.478018 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.478096 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.478498 (+ 402us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.478621 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:47:19.479725 21474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:47:19.483285 21355 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } attempt: 3
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:19.486553 21230 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } attempt: 3
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
I20260806 18:47:19.489761 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.480321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32796 (local address 127.1.245.125:41033)
0806 18:47:19.480457 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.480460 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.480472 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:47:19.480509 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.480511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.480548 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.480611 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.481039 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.481831 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.482922 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.483307 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.483404 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.485551 (+ 2147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.485555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.485558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.485584 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.487143 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.487626 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.487630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.487632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.487719 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.487987 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.487990 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.487992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.488164 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.488254 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.489478 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.489596 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:47:19.490135 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.480063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49078 (local address 127.1.245.126:35837)
0806 18:47:19.480167 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.480171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.480184 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:47:19.480218 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.480220 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.480260 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.480332 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.481054 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.481812 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.482922 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.483294 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.483404 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.486128 (+ 2724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.486132 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.486135 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.486159 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.488258 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.488691 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.488695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.488696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.488784 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.489103 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.489106 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.489108 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.489324 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.489529 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.489889 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.489999 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:47:19.490162 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.480210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:41033 (local address 127.0.0.1:32796)
0806 18:47:19.480314 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.480327 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.480385 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.480650 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.480653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.480806 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.480920 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.480927 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.481929 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.481932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.482769 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.482780 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.483520 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.483522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.483608 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.484063 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.484076 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.485414 (+ 1338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.487260 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.487277 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.487279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.487515 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.487825 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.487827 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.487828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.487898 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.489170 (+ 1272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.489173 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.489273 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.489888 (+ 615us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.490024 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 18:47:19.490976 21474 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042038, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
I20260806 18:47:19.493804 21361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.484061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35416)
0806 18:47:19.484207 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.484222 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.484311 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.484619 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.484622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.484798 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.484900 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.484907 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.485675 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.485678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.486451 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.486461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.487229 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.487233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.487323 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.487753 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.487768 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.489058 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.492087 (+ 3029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.492090 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.492092 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.492287 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.492800 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.492803 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.492805 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.492894 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.493338 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.493342 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.493438 (+ 96us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.493517 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.493641 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260806 18:47:19.494519 21477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.483716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.1.245.124:33375)
0806 18:47:19.483848 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.483853 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.484395 (+ 542us) server_negotiation.cc:407] Connection header received
0806 18:47:19.484435 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.484438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.484479 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.484587 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.485019 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.485550 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.486773 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.487102 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.487506 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.489182 (+ 1676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.489185 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.489188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.489211 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.491987 (+ 2776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.492374 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.492378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.492380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.492691 (+ 311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.493003 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.493007 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.493008 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.493208 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.494154 (+ 946us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.494227 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.494388 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:47:19.497068 21234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.486945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35422)
0806 18:47:19.487061 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.487074 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.487142 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.488876 (+ 1734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.488879 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.489039 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.489139 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.489146 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.490199 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.490203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.490923 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.490935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.491640 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.491644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.491721 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.492113 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.492127 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.493505 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.495534 (+ 2029us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.495537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.495539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.495785 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.496079 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.496082 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.496084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.496174 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.496591 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.496594 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.496685 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.496782 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.496913 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042038, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
I20260806 18:47:19.497797 21481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.487169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35422 (local address 127.1.245.124:33375)
0806 18:47:19.488650 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.488654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.488666 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:47:19.488695 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.488697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.488733 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.489240 (+ 507us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.489328 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.490080 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.491171 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.491519 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.491930 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.493635 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.493638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.493641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.493664 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.495415 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.495889 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.495892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.495894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.495974 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.496288 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.496291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.496294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.496485 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.497517 (+ 1032us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.497587 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.497671 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1418,"thread_start_us":108,"threads_started":1}
Aug 06 18:47:19 dist-test-slave-fgb3 krb5kdc[21109](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042039, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
I20260806 18:47:19.498339 21355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: Bootstrap starting.
I20260806 18:47:19.500553 21230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308: Bootstrap starting.
I20260806 18:47:19.501395 21355 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:19.502380 21355 log.cc:824] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:19.502725 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.492498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32804 (local address 127.1.245.125:41033)
0806 18:47:19.492594 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.492598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.492612 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:47:19.492765 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.492769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.492811 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.492857 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.493653 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.494306 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.495334 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.495606 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.495676 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.499293 (+ 3617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.499296 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.499299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.499323 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.500812 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.501279 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.501283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.501284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.501370 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.501696 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.501700 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.501702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.501913 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.502070 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.502482 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.502594 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:47:19.502847 21480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.491435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:33375 (local address 127.0.0.1:35432)
0806 18:47:19.491796 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:19.491810 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:19.491882 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:19.493043 (+ 1161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:19.493046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:19.493206 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:19.493305 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.493312 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.494510 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.494513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.495592 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:19.495602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.496575 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.496578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:19.496645 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.497032 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:19.497045 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:19.499115 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:19.501371 (+ 2256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.501374 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.501375 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.501590 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.501855 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:19.501858 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:19.501860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:19.501935 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:19.502411 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:19.502414 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:19.502496 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 18:47:19.502592 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 18:47:19.502711 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300}
I20260806 18:47:19.503283 21230 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:19.504076 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.491555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35432 (local address 127.1.245.124:33375)
0806 18:47:19.492727 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.492735 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.492747 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:47:19.492779 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.492782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.492820 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.492907 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.493413 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.494398 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.495700 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.496391 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.496486 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.499264 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.499268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.499271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.499296 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.501262 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.501691 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.501693 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.501695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.501771 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.502034 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.502037 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.502038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.502215 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.502336 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.503052 (+ 716us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.503684 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1106,"thread_start_us":112,"threads_started":1}
I20260806 18:47:19.504142 21230 log.cc:824] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:19.505198 21355 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: No bootstrap required, opened a new log
I20260806 18:47:19.506217 21230 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308: No bootstrap required, opened a new log
I20260806 18:47:19.506551 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:47:19.506870 21474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: Bootstrap starting.
I20260806 18:47:19.509361 21474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:19.509333 21355 raft_consensus.cc:348] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.509574 21355 raft_consensus.cc:374] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:19.509626 21355 raft_consensus.cc:729] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e047dbee558b4c63a59ae4efcc2b42e7, State: Initialized, Role: FOLLOWER
I20260806 18:47:19.510012 21230 raft_consensus.cc:348] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.510159 21474 log.cc:824] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:19.510217 21230 raft_consensus.cc:374] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:19.510125 21355 consensus_queue.cc:260] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.510262 21230 raft_consensus.cc:729] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f966f7c8cec74b69a02bff57e0c50308, State: Initialized, Role: FOLLOWER
I20260806 18:47:19.510927 21230 consensus_queue.cc:260] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.511435 21500 sys_catalog.cc:455] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.511556 21355 sys_catalog.cc:564] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:19.511646 21500 sys_catalog.cc:458] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:47:19.512197 21230 sys_catalog.cc:564] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:19.517364 21474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: No bootstrap required, opened a new log
W20260806 18:47:19.517472 21529 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:47:19.517530 21529 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 18:47:19.522492 21164 tablet.cc:2369] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:19.522149 21474 raft_consensus.cc:348] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.523242 21474 raft_consensus.cc:374] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:19.523360 21474 raft_consensus.cc:729] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 980e93640bf44fc5a993088a17c7de20, State: Initialized, Role: FOLLOWER
I20260806 18:47:19.523674 21365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.509493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.1.245.125:41033)
0806 18:47:19.509593 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.509597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.515135 (+ 5538us) server_negotiation.cc:407] Connection header received
0806 18:47:19.515243 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.515246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.515293 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.515367 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.516311 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.516996 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.518051 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.518386 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.518460 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.520199 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.520202 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.520205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.520230 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.521826 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.522334 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.522338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.522340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.522435 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.522742 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.522745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.522747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.522961 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.523070 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.523410 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.523526 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:47:19.523911 21474 consensus_queue.cc:260] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.520656 21240 master_runner.cc:186] Error getting master registration for 127.1.245.124:33375: OK, instance_id { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" instance_seqno: 1786042039456905 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:47:19.525238 21474 sys_catalog.cc:564] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:19.525439 21240 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:47:19.526568 21496 sys_catalog.cc:455] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.526695 21496 sys_catalog.cc:458] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:47:19.523512 21128 master_runner.cc:186] Error getting master registration for 127.1.245.124:33375: OK, instance_id { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" instance_seqno: 1786042039456905 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:47:19.527753 21128 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:47:19.528844 21235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.508808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49082 (local address 127.1.245.126:35837)
0806 18:47:19.508921 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.508925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.510629 (+ 1704us) server_negotiation.cc:407] Connection header received
0806 18:47:19.510827 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.510831 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.510881 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.510965 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.511852 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.512632 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.513714 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.514015 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.514109 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.525335 (+ 11226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.525340 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.525342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.525369 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.526889 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.527355 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.527359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.527361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.527446 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.527756 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.527760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.527762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.527977 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.528116 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.528369 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.528718 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":45}
W20260806 18:47:19.531431 21560 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:47:19.531808 21560 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260806 18:47:19.532575 21535 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:47:19.532641 21535 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:47:19.533754 21367 master_runner.cc:263] No leader master found from master 980e93640bf44fc5a993088a17c7de20
I20260806 18:47:19.533812 21367 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:47:19.534008 21540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.534282 21540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:47:19.535141 21482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:19.510089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35438 (local address 127.1.245.124:33375)
0806 18:47:19.510642 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:19.510646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:19.515435 (+ 4789us) server_negotiation.cc:407] Connection header received
0806 18:47:19.515538 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:19.515541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:19.515587 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:19.515653 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:19.518302 (+ 2649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.519062 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.520161 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:19.520528 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:19.520601 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:19.531380 (+ 10779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:19.531384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:19.531387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:19.531431 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:19.533094 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.533597 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.533601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.533603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.533704 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:19.534048 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:19.534051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:19.534053 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:19.534287 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:19.534472 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:19.534861 (+ 389us) server_negotiation.cc:299] Negotiation successful
0806 18:47:19.534994 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482}
W20260806 18:47:19.703292 21398 tablet.cc:2369] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:47:19.726686 21277 tablet.cc:2369] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:19.730412 21500 raft_consensus.cc:482] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:19.730584 21500 raft_consensus.cc:504] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.731536 21500 leader_election.cc:288] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e047dbee558b4c63a59ae4efcc2b42e7 (127.1.245.125:41033), 980e93640bf44fc5a993088a17c7de20 (127.1.245.124:33375)
I20260806 18:47:19.732026 21316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f966f7c8cec74b69a02bff57e0c50308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" is_pre_election: true
I20260806 18:47:19.732100 21431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f966f7c8cec74b69a02bff57e0c50308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "980e93640bf44fc5a993088a17c7de20" is_pre_election: true
I20260806 18:47:19.732343 21316 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f966f7c8cec74b69a02bff57e0c50308 in term 0.
I20260806 18:47:19.732424 21431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f966f7c8cec74b69a02bff57e0c50308 in term 0.
I20260806 18:47:19.732731 21158 leader_election.cc:302] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e047dbee558b4c63a59ae4efcc2b42e7, f966f7c8cec74b69a02bff57e0c50308; no voters:
I20260806 18:47:19.733007 21500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:47:19.733090 21500 raft_consensus.cc:482] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:47:19.733121 21500 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:19.734198 21500 raft_consensus.cc:504] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.734532 21500 leader_election.cc:288] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [CANDIDATE]: Term 1 election: Requested vote from peers e047dbee558b4c63a59ae4efcc2b42e7 (127.1.245.125:41033), 980e93640bf44fc5a993088a17c7de20 (127.1.245.124:33375)
I20260806 18:47:19.734856 21316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f966f7c8cec74b69a02bff57e0c50308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e047dbee558b4c63a59ae4efcc2b42e7"
I20260806 18:47:19.734997 21316 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:19.735031 21431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f966f7c8cec74b69a02bff57e0c50308" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "980e93640bf44fc5a993088a17c7de20"
I20260806 18:47:19.735132 21431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:19.736068 21316 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f966f7c8cec74b69a02bff57e0c50308 in term 1.
I20260806 18:47:19.736106 21431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f966f7c8cec74b69a02bff57e0c50308 in term 1.
I20260806 18:47:19.736373 21158 leader_election.cc:302] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e047dbee558b4c63a59ae4efcc2b42e7, f966f7c8cec74b69a02bff57e0c50308; no voters:
I20260806 18:47:19.736538 21500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:19.736788 21500 raft_consensus.cc:686] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [term 1 LEADER]: Becoming Leader. State: Replica: f966f7c8cec74b69a02bff57e0c50308, State: Running, Role: LEADER
I20260806 18:47:19.737138 21500 consensus_queue.cc:237] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } }
I20260806 18:47:19.739053 21563 sys_catalog.cc:455] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f966f7c8cec74b69a02bff57e0c50308. Latest consensus state: current_term: 1 leader_uuid: "f966f7c8cec74b69a02bff57e0c50308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.739154 21563 sys_catalog.cc:458] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:19.739497 21567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:47:19.740360 21567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:47:19.744972 21431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [term 1 FOLLOWER]: Refusing update from remote peer f966f7c8cec74b69a02bff57e0c50308: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:19.745007 21316 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [term 1 FOLLOWER]: Refusing update from remote peer f966f7c8cec74b69a02bff57e0c50308: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:19.745648 21500 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:19.745867 21563 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:19.750064 21540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f966f7c8cec74b69a02bff57e0c50308. Latest consensus state: current_term: 1 leader_uuid: "f966f7c8cec74b69a02bff57e0c50308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.750201 21540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:47:19.750133 21496 sys_catalog.cc:455] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f966f7c8cec74b69a02bff57e0c50308. Latest consensus state: current_term: 1 leader_uuid: "f966f7c8cec74b69a02bff57e0c50308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.750264 21496 sys_catalog.cc:458] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:47:19.752563 21500 sys_catalog.cc:455] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f966f7c8cec74b69a02bff57e0c50308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.752682 21500 sys_catalog.cc:458] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:19.752831 21570 mvcc.cc:205] Tried to move back new op lower bound from 7315628194788970496 to 7315628194765692928. Current Snapshot: MvccSnapshot[applied={T|T < 7315628194788970496}]
I20260806 18:47:19.753034 21540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f966f7c8cec74b69a02bff57e0c50308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.753142 21540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:47:19.753176 21500 sys_catalog.cc:455] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f966f7c8cec74b69a02bff57e0c50308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.753275 21500 sys_catalog.cc:458] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:19.754340 21496 sys_catalog.cc:455] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f966f7c8cec74b69a02bff57e0c50308" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f966f7c8cec74b69a02bff57e0c50308" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35837 } } peers { permanent_uuid: "e047dbee558b4c63a59ae4efcc2b42e7" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 41033 } } peers { permanent_uuid: "980e93640bf44fc5a993088a17c7de20" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 33375 } } }
I20260806 18:47:19.754458 21496 sys_catalog.cc:458] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:47:19.754539 21567 catalog_manager.cc:1347] Generated new cluster ID: ed921e467bc647ddbb7fd86e309ee7f8
I20260806 18:47:19.754597 21567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:47:19.777727 21567 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:47:19.778985 21567 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:47:19.793860 21567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f966f7c8cec74b69a02bff57e0c50308: Generated new TSK 0
I20260806 18:47:19.794865 21567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:47:20.518965 21529 catalog_manager.cc:1259] Loaded cluster ID: ed921e467bc647ddbb7fd86e309ee7f8
I20260806 18:47:20.519053 21529 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: loading cluster ID for follower catalog manager: success
I20260806 18:47:20.520684 21529 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: acquiring CA information for follower catalog manager: success
I20260806 18:47:20.521842 21529 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e047dbee558b4c63a59ae4efcc2b42e7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:47:20.533468 21560 catalog_manager.cc:1259] Loaded cluster ID: ed921e467bc647ddbb7fd86e309ee7f8
I20260806 18:47:20.533531 21560 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: loading cluster ID for follower catalog manager: success
I20260806 18:47:20.535089 21560 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: acquiring CA information for follower catalog manager: success
I20260806 18:47:20.536178 21560 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 980e93640bf44fc5a993088a17c7de20: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:47:20.869609 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 21128
I20260806 18:47:20.882962 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 21240
I20260806 18:47:20.896515 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 21367
2026-08-06T18:47:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3299 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:47:20 dist-test-slave-fgb3 krb5kdc[21589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:47:20 dist-test-slave-fgb3 krb5kdc[21589](info): set up 2 sockets
Aug 06 18:47:20 dist-test-slave-fgb3 krb5kdc[21589](info): commencing operation
krb5kdc: starting...
W20260806 18:47:22.024235 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:47:22 dist-test-slave-fgb3 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:47:22.148998 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35935
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35935
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:22.259176 21610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:22.259425 21610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:22.259464 21610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:22.263278 21610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:22.263324 21610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:22.263341 21610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:22.263355 21610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:22.263370 21610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:22.268370 21610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35935
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:22.269291 21610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:22.270215 21610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:22.278657 21618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:22.278676 21622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:22.278734 21610 server_base.cc:1034] running on GCE node
W20260806 18:47:22.278681 21619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:22.279438 21610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:22.280516 21610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:22.281697 21610 hybrid_clock.cc:648] HybridClock initialized: now 1786042042281671 us; error 36 us; skew 500 ppm
Aug 06 18:47:22 dist-test-slave-fgb3 krb5kdc[21589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042042, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:22.284436 21610 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:47:22.285285 21610 webserver.cc:460] Webserver started at http://127.1.245.126:43141/ using document root <none> and password file <none>
I20260806 18:47:22.285810 21610 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:22.285867 21610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:22.286031 21610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:22.288101 21610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "faaa2e885030400dbbc183a1b7d4546d"
format_stamp: "Formatted at 2026-08-06 18:47:22 on dist-test-slave-fgb3"
server_key: "4cc9a4b73ea36a50e2524c766a2ac51c"
server_key_iv: "707ff0da0f12bab9236ce2d2f5651857"
server_key_version: "encryptionkey@0"
I20260806 18:47:22.288614 21610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "faaa2e885030400dbbc183a1b7d4546d"
format_stamp: "Formatted at 2026-08-06 18:47:22 on dist-test-slave-fgb3"
server_key: "4cc9a4b73ea36a50e2524c766a2ac51c"
server_key_iv: "707ff0da0f12bab9236ce2d2f5651857"
server_key_version: "encryptionkey@0"
I20260806 18:47:22.292430 21610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:47:22.295027 21633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:22.296324 21610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:47:22.296452 21610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "faaa2e885030400dbbc183a1b7d4546d"
format_stamp: "Formatted at 2026-08-06 18:47:22 on dist-test-slave-fgb3"
server_key: "4cc9a4b73ea36a50e2524c766a2ac51c"
server_key_iv: "707ff0da0f12bab9236ce2d2f5651857"
server_key_version: "encryptionkey@0"
I20260806 18:47:22.296582 21610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:22.328485 21610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:22.331679 21610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:22.331874 21610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:22.339107 21610 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:35935
I20260806 18:47:22.339123 21713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:35935 every 8 connection(s)
I20260806 18:47:22.340122 21610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:47:22.343057 21714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:22.345471 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 21610
I20260806 18:47:22.345604 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:47:22.345916 2005 external_mini_cluster.cc:1428] Setting key 66e38e9d1489407ac878665c4000ef36
I20260806 18:47:22.349380 21714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d: Bootstrap starting.
I20260806 18:47:22.351936 21714 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:22.352744 21714 log.cc:824] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:47:22 dist-test-slave-fgb3 krb5kdc[21589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042042, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:22.354915 21714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d: No bootstrap required, opened a new log
I20260806 18:47:22.359304 21714 raft_consensus.cc:348] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaa2e885030400dbbc183a1b7d4546d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35935 } }
I20260806 18:47:22.359599 21714 raft_consensus.cc:374] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:22.359658 21714 raft_consensus.cc:729] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: faaa2e885030400dbbc183a1b7d4546d, State: Initialized, Role: FOLLOWER
I20260806 18:47:22.360137 21714 consensus_queue.cc:260] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaa2e885030400dbbc183a1b7d4546d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35935 } }
I20260806 18:47:22.360296 21714 raft_consensus.cc:388] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:47:22.360363 21714 raft_consensus.cc:482] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:47:22.360400 21718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:22.347256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.1.245.126:35935)
0806 18:47:22.347767 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:22.347789 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:22.347833 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:47:22.348582 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:22.348601 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:22.348940 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:22.349215 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:22.350073 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:22.351106 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:22.352225 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:22.352684 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:22.352810 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:22.355428 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:22.355435 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:22.355451 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:22.355482 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:22.357322 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:22.357727 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:22.357734 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:22.357739 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:22.357880 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:22.358113 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:22.358117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:22.358118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:22.358673 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:22.358875 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:22.359113 (+ 238us) server_negotiation.cc:299] Negotiation successful
0806 18:47:22.359409 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":134,"threads_started":1}
I20260806 18:47:22.360472 21714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:22.361388 21714 raft_consensus.cc:504] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaa2e885030400dbbc183a1b7d4546d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35935 } }
I20260806 18:47:22.361728 21714 leader_election.cc:302] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: faaa2e885030400dbbc183a1b7d4546d; no voters:
I20260806 18:47:22.362025 21714 leader_election.cc:288] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:47:22.362136 21720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:22.362413 21720 raft_consensus.cc:686] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [term 1 LEADER]: Becoming Leader. State: Replica: faaa2e885030400dbbc183a1b7d4546d, State: Running, Role: LEADER
I20260806 18:47:22.362746 21720 consensus_queue.cc:237] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaa2e885030400dbbc183a1b7d4546d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35935 } }
I20260806 18:47:22.363189 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:47:22.363430 21714 sys_catalog.cc:564] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:22.364360 21722 sys_catalog.cc:455] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "faaa2e885030400dbbc183a1b7d4546d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaa2e885030400dbbc183a1b7d4546d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35935 } } }
I20260806 18:47:22.364482 21722 sys_catalog.cc:458] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:22.364921 21721 sys_catalog.cc:455] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [sys.catalog]: SysCatalogTable state changed. Reason: New leader faaa2e885030400dbbc183a1b7d4546d. Latest consensus state: current_term: 1 leader_uuid: "faaa2e885030400dbbc183a1b7d4546d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "faaa2e885030400dbbc183a1b7d4546d" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35935 } } }
I20260806 18:47:22.365089 21721 sys_catalog.cc:458] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:22.365763 21737 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:47:22.368386 21737 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:47:22.373427 21737 catalog_manager.cc:1347] Generated new cluster ID: a77775d2ccc44de1ac913a09d1c4455d
I20260806 18:47:22.373495 21737 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:47:22.389405 21737 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:47:22.390491 21737 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:47:22.396613 21737 catalog_manager.cc:5868] T 00000000000000000000000000000000 P faaa2e885030400dbbc183a1b7d4546d: Generated new TSK 0
I20260806 18:47:22.397308 21737 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:47:22.419080 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 21610
2026-08-06T18:47:22Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1518 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:47:22 dist-test-slave-fgb3 krb5kdc[21764](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:47:22 dist-test-slave-fgb3 krb5kdc[21764](info): set up 2 sockets
Aug 06 18:47:22 dist-test-slave-fgb3 krb5kdc[21764](info): commencing operation
krb5kdc: starting...
W20260806 18:47:23.536818 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.093s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:47:23 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:47:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:47:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:47:23.661877 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:46209
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:46209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:23.778141 21783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:23.778473 21783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:23.778517 21783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:23.782644 21783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:23.782720 21783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:23.782740 21783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:23.782756 21783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:23.782770 21783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:23.788271 21783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:46209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:46209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21783
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:23.789398 21783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:23.790475 21783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:47:23.799255 21783 server_base.cc:1034] running on GCE node
W20260806 18:47:23.799166 21792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:23.799275 21795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:23.799171 21793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:23.799996 21783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:23.801277 21783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:23.802431 21783 hybrid_clock.cc:648] HybridClock initialized: now 1786042043802406 us; error 40 us; skew 500 ppm
Aug 06 18:47:23 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042043, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:23.808735 21783 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:47:23.809782 21783 webserver.cc:460] Webserver started at http://127.1.245.126:46791/ using document root <none> and password file <none>
I20260806 18:47:23.810410 21783 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:23.810488 21783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:23.810709 21783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:23.812862 21783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "fdd9089a3f964ac79b3fe24f90c57be5"
format_stamp: "Formatted at 2026-08-06 18:47:23 on dist-test-slave-fgb3"
server_key: "064b15d794c6e7ef83059d13476345f9"
server_key_iv: "827417438c1a7a96fdcb2d5f3a02a95e"
server_key_version: "encryptionkey@0"
I20260806 18:47:23.813624 21783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "fdd9089a3f964ac79b3fe24f90c57be5"
format_stamp: "Formatted at 2026-08-06 18:47:23 on dist-test-slave-fgb3"
server_key: "064b15d794c6e7ef83059d13476345f9"
server_key_iv: "827417438c1a7a96fdcb2d5f3a02a95e"
server_key_version: "encryptionkey@0"
I20260806 18:47:23.817572 21783 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:47:23.820098 21803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:23.821291 21783 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:47:23.821434 21783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "fdd9089a3f964ac79b3fe24f90c57be5"
format_stamp: "Formatted at 2026-08-06 18:47:23 on dist-test-slave-fgb3"
server_key: "064b15d794c6e7ef83059d13476345f9"
server_key_iv: "827417438c1a7a96fdcb2d5f3a02a95e"
server_key_version: "encryptionkey@0"
I20260806 18:47:23.821529 21783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:23.839069 21783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:23.842187 21783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:23.842411 21783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:23.849272 21783 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:46209
I20260806 18:47:23.849539 21882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:46209 every 8 connection(s)
I20260806 18:47:23.850253 21783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:47:23.853250 21884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:23.859197 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 21783
I20260806 18:47:23.859333 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:47:23.859315 21884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Bootstrap starting.
I20260806 18:47:23.859632 2005 external_mini_cluster.cc:1428] Setting key 2c613ffdbeeccdc5a92fb7396d496fd3
I20260806 18:47:23.861937 21884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:23.862730 21884 log.cc:824] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:23.864751 21884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: No bootstrap required, opened a new log
Aug 06 18:47:23 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042043, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:23.869194 21884 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:23.869506 21884 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:23.869586 21884 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdd9089a3f964ac79b3fe24f90c57be5, State: Initialized, Role: FOLLOWER
I20260806 18:47:23.870154 21884 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:23.870359 21884 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:47:23.870433 21884 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:47:23.870538 21884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:23.871651 21884 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:23.872030 21884 leader_election.cc:302] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdd9089a3f964ac79b3fe24f90c57be5; no voters:
I20260806 18:47:23.872387 21884 leader_election.cc:288] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:47:23.872534 21890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:23.872782 21890 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 1 LEADER]: Becoming Leader. State: Replica: fdd9089a3f964ac79b3fe24f90c57be5, State: Running, Role: LEADER
I20260806 18:47:23.873186 21890 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:23.873708 21884 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:23.874760 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:23.860976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.1.245.126:46209)
0806 18:47:23.861392 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:23.861408 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:23.861444 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:47:23.862117 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:23.862136 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:23.862506 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:23.862782 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:23.863616 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:23.864618 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:23.865719 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:23.866197 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:23.866342 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:23.869112 (+ 2770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:23.869118 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:23.869132 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:23.869161 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:23.871088 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:23.871508 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:23.871514 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:23.871521 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:23.871671 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:23.871939 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:23.871942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:23.871944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:23.872520 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:23.872712 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:23.873412 (+ 700us) server_negotiation.cc:299] Negotiation successful
0806 18:47:23.873722 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":128,"threads_started":1}
I20260806 18:47:23.874959 21893 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } } }
I20260806 18:47:23.875063 21893 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:23.875273 21893 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdd9089a3f964ac79b3fe24f90c57be5. Latest consensus state: current_term: 1 leader_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } } }
I20260806 18:47:23.875349 21893 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: This master's current role is: LEADER
W20260806 18:47:23.878482 21913 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:47:23.878553 21913 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:47:23.878611 21901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:47:23.879400 21901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:47:23.884519 21901 catalog_manager.cc:1347] Generated new cluster ID: 3931e0403885482abf8dfccacfb5f62b
I20260806 18:47:23.884595 21901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:47:23.907003 21901 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:47:23.907970 21901 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:47:23.913018 21901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Generated new TSK 0
I20260806 18:47:23.913686 21901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:47:23.946884 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:46209
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:24.061280 21921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:24.061537 21921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:24.061579 21921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:24.065625 21921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:24.065687 21921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:24.065748 21921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:47:24.071008 21921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:24.072053 21921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:24.073100 21921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:24.082624 21929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:24.082858 21921 server_base.cc:1034] running on GCE node
W20260806 18:47:24.082595 21927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:24.082592 21934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:24.083415 21921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:24.084028 21921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:24.085181 21921 hybrid_clock.cc:648] HybridClock initialized: now 1786042044085180 us; error 48 us; skew 500 ppm
Aug 06 18:47:24 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042044, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:24.087875 21921 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:47:24.088737 21921 webserver.cc:460] Webserver started at http://127.1.245.65:38835/ using document root <none> and password file <none>
I20260806 18:47:24.089262 21921 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:24.089319 21921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:24.089493 21921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:24.091557 21921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "fbf499ccd73a4dfd989d28f072aa72c3"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "48b8ca7f816276b85748ffba037bb9f2"
server_key_iv: "7de051148e1ff69d28397496e0a0fd6a"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.092198 21921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "fbf499ccd73a4dfd989d28f072aa72c3"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "48b8ca7f816276b85748ffba037bb9f2"
server_key_iv: "7de051148e1ff69d28397496e0a0fd6a"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.095890 21921 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:47:24.098263 21944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.099539 21921 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:47:24.099646 21921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "fbf499ccd73a4dfd989d28f072aa72c3"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "48b8ca7f816276b85748ffba037bb9f2"
server_key_iv: "7de051148e1ff69d28397496e0a0fd6a"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.099747 21921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:24.126428 21921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:24.129530 21921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:24.129712 21921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:24.130360 21921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:24.131217 21921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:24.131265 21921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.131316 21921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:24.131345 21921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.140022 21921 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:44671
I20260806 18:47:24.140048 22138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:44671 every 8 connection(s)
I20260806 18:47:24.140950 21921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:47:24.144340 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 21921
I20260806 18:47:24.144497 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:47:24.144779 2005 external_mini_cluster.cc:1428] Setting key 6292e055ab485c927d62d590295193d8
Aug 06 18:47:24 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042044, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:47:24.155157 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.142783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:43707 (local address 127.1.245.126:46209)
0806 18:47:24.142936 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.142941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.143861 (+ 920us) server_negotiation.cc:407] Connection header received
0806 18:47:24.145378 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.145382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.145443 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.145516 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:24.146945 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.147481 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.148774 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.149014 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.149106 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.151581 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:24.151584 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:24.151586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:24.151612 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:24.152991 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:24.153496 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:24.153498 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:24.153500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:24.153581 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:24.153909 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:24.153912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:24.153914 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:24.154136 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:24.154229 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.154838 (+ 609us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.154976 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:47:24.155997 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.143230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:43707)
0806 18:47:24.143686 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.143731 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:24.145153 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:24.145724 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:24.145731 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:24.146093 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:24.146782 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.146803 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.147606 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.147609 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.148654 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.148665 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.149160 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.149164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.149328 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.149893 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:24.149911 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:24.151415 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:24.153106 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:24.153112 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:24.153130 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:24.153399 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:24.153664 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:24.153667 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:24.153669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:24.153817 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:24.154240 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:24.154247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:24.154563 (+ 316us) client_negotiation.cc:769] Sending connection context
0806 18:47:24.154740 (+ 177us) client_negotiation.cc:240] Negotiation successful
0806 18:47:24.155107 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:47:24.157279 22139 heartbeater.cc:347] Connected to a master server at 127.1.245.126:46209
I20260806 18:47:24.157555 22139 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:24.158185 22139 heartbeater.cc:511] Master 127.1.245.126:46209 requested a full tablet report, sending...
I20260806 18:47:24.160060 21832 ts_manager.cc:194] Registered new tserver with Master: fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671)
I20260806 18:47:24.161221 21832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:43707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:47:24.173602 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:46209
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:24.287366 22147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:24.287626 22147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:24.287667 22147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:24.291718 22147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:24.291787 22147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:24.291846 22147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:47:24.297104 22147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:24.298197 22147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:24.299309 22147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:24.309329 22154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:24.309477 22147 server_base.cc:1034] running on GCE node
W20260806 18:47:24.309305 22159 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:24.309315 22156 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:24.309931 22147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:24.310611 22147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:24.311774 22147 hybrid_clock.cc:648] HybridClock initialized: now 1786042044311758 us; error 35 us; skew 500 ppm
Aug 06 18:47:24 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042044, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:24.314608 22147 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:47:24.315508 22147 webserver.cc:460] Webserver started at http://127.1.245.66:41045/ using document root <none> and password file <none>
I20260806 18:47:24.316069 22147 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:24.316136 22147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:24.316345 22147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:24.318456 22147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "40020e316fb34301b76b4b51ef57ce23"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "d6bf1e9ea29078a5b218dbd43bf93b76"
server_key_iv: "c32eb7416063d6b71505b8504ea6212c"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.319025 22147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "40020e316fb34301b76b4b51ef57ce23"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "d6bf1e9ea29078a5b218dbd43bf93b76"
server_key_iv: "c32eb7416063d6b71505b8504ea6212c"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.322799 22147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:47:24.325292 22173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.326584 22147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:47:24.326694 22147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "40020e316fb34301b76b4b51ef57ce23"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "d6bf1e9ea29078a5b218dbd43bf93b76"
server_key_iv: "c32eb7416063d6b71505b8504ea6212c"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.326790 22147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:24.346216 22147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:24.349403 22147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:24.349586 22147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:24.350189 22147 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:24.351045 22147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:24.351092 22147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.351146 22147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:24.351173 22147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.361099 22147 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:33803
I20260806 18:47:24.361121 22344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:33803 every 8 connection(s)
I20260806 18:47:24.361991 22147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:47:24.371130 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 22147
I20260806 18:47:24.371243 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
Aug 06 18:47:24 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042044, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:24.371548 2005 external_mini_cluster.cc:1428] Setting key fc9534b488ba528f9832f1fe11d3115c
I20260806 18:47:24.376255 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.363873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:37111 (local address 127.1.245.126:46209)
0806 18:47:24.364029 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.364034 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.364916 (+ 882us) server_negotiation.cc:407] Connection header received
0806 18:47:24.366159 (+ 1243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.366162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.366229 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.366348 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:24.367506 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.368061 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.369146 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.369381 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.369470 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.372037 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:24.372041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:24.372044 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:24.372075 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:24.373772 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:24.374423 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:24.374428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:24.374430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:24.374526 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:24.374904 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:24.374908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:24.374910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:24.375198 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:24.375308 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.375891 (+ 583us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.376062 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:47:24.377101 22350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.364292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:37111)
0806 18:47:24.364756 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.364800 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:24.365948 (+ 1148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:24.366498 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:24.366505 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:24.366865 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:24.367363 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.367376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.368154 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.368156 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.369027 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.369035 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.369510 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.369512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.369649 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.370205 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:24.370222 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:24.371839 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:24.373906 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:24.373913 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:24.373930 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:24.374296 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:24.374619 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:24.374623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:24.374625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:24.374793 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:24.375319 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:24.375327 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:24.375597 (+ 270us) client_negotiation.cc:769] Sending connection context
0806 18:47:24.375784 (+ 187us) client_negotiation.cc:240] Negotiation successful
0806 18:47:24.376140 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":121,"threads_started":1}
I20260806 18:47:24.378331 22347 heartbeater.cc:347] Connected to a master server at 127.1.245.126:46209
I20260806 18:47:24.378598 22347 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:24.379098 22347 heartbeater.cc:511] Master 127.1.245.126:46209 requested a full tablet report, sending...
I20260806 18:47:24.380201 21832 ts_manager.cc:194] Registered new tserver with Master: 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803)
I20260806 18:47:24.380977 21832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:37111
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:47:24.405689 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:46209
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:24.517452 22354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:24.517702 22354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:24.517760 22354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:24.521807 22354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:24.521869 22354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:24.521948 22354 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:47:24.527206 22354 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:24.528204 22354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:24.529186 22354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:24.538915 22360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:24.538909 22364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:24.538905 22361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:24.539243 22354 server_base.cc:1034] running on GCE node
I20260806 18:47:24.539721 22354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:24.540406 22354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:24.541572 22354 hybrid_clock.cc:648] HybridClock initialized: now 1786042044541546 us; error 36 us; skew 500 ppm
Aug 06 18:47:24 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042044, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:24.544390 22354 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:47:24.545375 22354 webserver.cc:460] Webserver started at http://127.1.245.67:41171/ using document root <none> and password file <none>
I20260806 18:47:24.545941 22354 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:24.546010 22354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:24.546206 22354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:24.548383 22354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "e02f5bd902a34b68b2fe9efee6b32a48"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "082e5ec642df46f17a3c471c38a2cdf3"
server_key_iv: "8722c6b2b76bd7ec716fbe4ed5034f5e"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.548952 22354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "e02f5bd902a34b68b2fe9efee6b32a48"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "082e5ec642df46f17a3c471c38a2cdf3"
server_key_iv: "8722c6b2b76bd7ec716fbe4ed5034f5e"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.552901 22354 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 18:47:24.555610 22373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.557016 22354 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:47:24.557142 22354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "e02f5bd902a34b68b2fe9efee6b32a48"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "082e5ec642df46f17a3c471c38a2cdf3"
server_key_iv: "8722c6b2b76bd7ec716fbe4ed5034f5e"
server_key_version: "encryptionkey@0"
I20260806 18:47:24.557248 22354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:24.566674 22354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:24.569517 22354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:24.569715 22354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:24.570371 22354 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:24.571245 22354 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:24.571303 22354 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.571357 22354 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:24.571384 22354 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:24.583355 22354 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:35763
I20260806 18:47:24.583420 22542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:35763 every 8 connection(s)
I20260806 18:47:24.584438 22354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
Aug 06 18:47:24 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042044, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:24.593675 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 22354
I20260806 18:47:24.593796 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:47:24.594108 2005 external_mini_cluster.cc:1428] Setting key 220474ec68f56cdb50166d361288e7d9
I20260806 18:47:24.597556 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.586233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:36875 (local address 127.1.245.126:46209)
0806 18:47:24.586440 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.586445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.587286 (+ 841us) server_negotiation.cc:407] Connection header received
0806 18:47:24.588496 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.588499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.588547 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.588619 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:24.589742 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.590268 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.591340 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.591549 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.591621 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.594135 (+ 2514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:24.594138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:24.594140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:24.594170 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:24.595575 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:24.596073 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:24.596077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:24.596078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:24.596159 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:24.596534 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:24.596538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:24.596540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:24.596765 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:24.596847 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.597315 (+ 468us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.597436 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:47:24.598500 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.586653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:36875)
0806 18:47:24.587127 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.587171 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:24.588301 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:24.588775 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:24.588781 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:24.589132 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:24.589626 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.589640 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.590377 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.590379 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.591243 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.591252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.591663 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.591665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.591804 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.592377 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:24.592393 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:24.593958 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:24.595688 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:24.595695 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:24.595711 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:24.595977 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:24.596242 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:24.596245 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:24.596248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:24.596449 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:24.596852 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:24.596857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:24.597120 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 18:47:24.597280 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 18:47:24.597552 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":129,"threads_started":1}
I20260806 18:47:24.599599 22543 heartbeater.cc:347] Connected to a master server at 127.1.245.126:46209
I20260806 18:47:24.599838 22543 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:24.600351 22543 heartbeater.cc:511] Master 127.1.245.126:46209 requested a full tablet report, sending...
I20260806 18:47:24.601343 21832 ts_manager.cc:194] Registered new tserver with Master: e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763)
I20260806 18:47:24.601902 21832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:36875
I20260806 18:47:24.608165 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:47:24.617074 21887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.610008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.1.245.126:46209)
0806 18:47:24.610123 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.610127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.610299 (+ 172us) server_negotiation.cc:407] Connection header received
0806 18:47:24.610479 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.610483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.610533 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.610599 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:24.611294 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.611818 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.612624 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.612847 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.612933 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.614245 (+ 1312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:24.614249 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:24.614252 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:24.614280 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:24.615446 (+ 1166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:24.615835 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:24.615839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:24.615841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:24.615929 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:24.616176 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:24.616179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:24.616181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:24.616427 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:24.616551 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.616785 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.616911 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:47:24.619879 21832 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:47:24.622953 21832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:47:24.637676 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.631562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43466 (local address 127.1.245.65:44671)
0806 18:47:24.631899 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.631917 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.631963 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:47:24.632260 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.632267 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.632485 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.632648 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:24.633192 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.634268 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.636393 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.636955 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.637205 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.637359 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.637412 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.637511 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":119,"threads_started":1}
I20260806 18:47:24.637701 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.632460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58402 (local address 127.1.245.66:33803)
0806 18:47:24.633016 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.633029 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.633052 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:47:24.633129 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.633134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.633314 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.633472 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:24.633698 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.634855 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.636205 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.636952 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.637041 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.637192 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.637472 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.637563 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":101,"threads_started":1}
I20260806 18:47:24.637689 22562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.631440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:44671 (local address 127.0.0.1:43466)
0806 18:47:24.631824 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.631888 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:24.632037 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:24.632859 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:24.632866 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:24.632915 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:24.633058 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.633066 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.634423 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.634426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.636285 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.636293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.637078 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.637081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.637228 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.637277 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 18:47:24.637380 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 18:47:24.637541 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":44,"thread_start_us":96,"threads_started":1}
I20260806 18:47:24.638001 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.632375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.1.245.67:35763)
0806 18:47:24.633341 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.633352 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.633375 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:47:24.633447 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.633451 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.633621 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.633768 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:24.633922 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.635032 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.636099 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.636922 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.637042 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.637192 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.637767 (+ 575us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.637862 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":866,"thread_start_us":101,"threads_started":1}
I20260806 18:47:24.638610 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.632038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:33803 (local address 127.0.0.1:58402)
0806 18:47:24.632543 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.632557 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:24.632646 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:24.633452 (+ 806us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:24.633456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:24.633478 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:24.633572 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.633577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.634999 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.635002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.636085 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.636096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.637256 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.637259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.637326 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.637345 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:47:24.637418 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:47:24.638490 (+ 1072us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":43,"thread_start_us":97,"threads_started":1}
I20260806 18:47:24.638912 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.632215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35763 (local address 127.0.0.1:42970)
0806 18:47:24.632792 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.632801 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:24.632864 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:24.633733 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:24.633735 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:24.633750 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:24.633832 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.633837 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.635151 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.635155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.635984 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.635993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.637059 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.637063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.637513 (+ 450us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.637534 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:47:24.637597 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:47:24.637659 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":40,"thread_start_us":116,"threads_started":1}
I20260806 18:47:24.641062 22244 tablet_service.cc:1467] Processing CreateTablet for tablet 52de4ebe879443499e3a8364cba00ea5 (DEFAULT_TABLE table=test-table [id=a87aa70f700c4976a11954fcc1a9d61a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:24.641101 22023 tablet_service.cc:1467] Processing CreateTablet for tablet 52de4ebe879443499e3a8364cba00ea5 (DEFAULT_TABLE table=test-table [id=a87aa70f700c4976a11954fcc1a9d61a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:24.641398 22447 tablet_service.cc:1467] Processing CreateTablet for tablet 52de4ebe879443499e3a8364cba00ea5 (DEFAULT_TABLE table=test-table [id=a87aa70f700c4976a11954fcc1a9d61a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:24.642135 22244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52de4ebe879443499e3a8364cba00ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:24.642181 22023 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52de4ebe879443499e3a8364cba00ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:24.642247 22447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52de4ebe879443499e3a8364cba00ea5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:24.648751 22569 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Bootstrap starting.
I20260806 18:47:24.649735 22570 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Bootstrap starting.
I20260806 18:47:24.650679 22573 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Bootstrap starting.
I20260806 18:47:24.651290 22569 tablet_bootstrap.cc:654] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:24.651818 22570 tablet_bootstrap.cc:654] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:24.652170 22569 log.cc:824] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:24.653074 22570 log.cc:824] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:24.653257 22573 tablet_bootstrap.cc:654] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:24.654071 22573 log.cc:824] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:24.654156 22569 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: No bootstrap required, opened a new log
I20260806 18:47:24.654379 22569 ts_tablet_manager.cc:1397] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:47:24.655030 22570 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: No bootstrap required, opened a new log
I20260806 18:47:24.655215 22570 ts_tablet_manager.cc:1397] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:47:24.656046 22573 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: No bootstrap required, opened a new log
I20260806 18:47:24.656229 22573 ts_tablet_manager.cc:1397] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260806 18:47:24.657564 22569 raft_consensus.cc:348] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.657794 22569 raft_consensus.cc:374] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:24.657850 22569 raft_consensus.cc:729] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40020e316fb34301b76b4b51ef57ce23, State: Initialized, Role: FOLLOWER
I20260806 18:47:24.658277 22569 consensus_queue.cc:260] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.658999 22347 heartbeater.cc:503] Master 127.1.245.126:46209 was elected leader, sending a full tablet report...
I20260806 18:47:24.659015 22570 raft_consensus.cc:348] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.659262 22569 ts_tablet_manager.cc:1428] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:47:24.659315 22570 raft_consensus.cc:374] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:24.659380 22570 raft_consensus.cc:729] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbf499ccd73a4dfd989d28f072aa72c3, State: Initialized, Role: FOLLOWER
I20260806 18:47:24.659795 22570 consensus_queue.cc:260] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.660440 22139 heartbeater.cc:503] Master 127.1.245.126:46209 was elected leader, sending a full tablet report...
I20260806 18:47:24.660734 22570 ts_tablet_manager.cc:1428] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:47:24.661235 22573 raft_consensus.cc:348] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.661491 22573 raft_consensus.cc:374] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:24.661543 22573 raft_consensus.cc:729] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02f5bd902a34b68b2fe9efee6b32a48, State: Initialized, Role: FOLLOWER
I20260806 18:47:24.661945 22573 consensus_queue.cc:260] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.662920 22573 ts_tablet_manager.cc:1428] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260806 18:47:24.663237 22543 heartbeater.cc:503] Master 127.1.245.126:46209 was elected leader, sending a full tablet report...
I20260806 18:47:24.827488 22584 raft_consensus.cc:482] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:24.827705 22584 raft_consensus.cc:504] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.828914 22584 leader_election.cc:288] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671), 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803)
I20260806 18:47:24.832587 22563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.829236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:49931 (local address 127.1.245.65:44671)
0806 18:47:24.829381 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.829386 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.829413 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:47:24.829548 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.829552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.829607 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.829685 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:24.829962 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.830617 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.831673 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.832079 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.832182 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.832238 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.832396 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.832469 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260806 18:47:24.832587 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.829135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:44671 (local address 127.1.245.67:49931)
0806 18:47:24.829290 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.829306 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:24.829431 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:24.829708 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:24.829711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:24.829725 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:24.829840 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.829845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.830721 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.830724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.831550 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.831561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.832186 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.832190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.832289 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.832307 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:47:24.832380 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:47:24.832451 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":24}
I20260806 18:47:24.832695 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.829416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:51629 (local address 127.1.245.66:33803)
0806 18:47:24.829548 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.829552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.829673 (+ 121us) server_negotiation.cc:407] Connection header received
0806 18:47:24.829800 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.829802 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.829844 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.829924 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:24.830159 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.830723 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.831821 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.832203 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.832308 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.832358 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.832519 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.832579 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:47:24.832741 22585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.829169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:33803 (local address 127.1.245.67:51629)
0806 18:47:24.829593 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.829603 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:24.829694 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:24.829960 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:24.829962 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:24.829979 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:24.830074 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.830080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.830837 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.830841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.831704 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:24.831714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.832308 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.832311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:24.832409 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.832432 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:47:24.832496 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:47:24.832566 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":33,"thread_start_us":116,"threads_started":1}
I20260806 18:47:24.833393 22057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52de4ebe879443499e3a8364cba00ea5" candidate_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" is_pre_election: true
I20260806 18:47:24.833395 22279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52de4ebe879443499e3a8364cba00ea5" candidate_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40020e316fb34301b76b4b51ef57ce23" is_pre_election: true
I20260806 18:47:24.833667 22057 raft_consensus.cc:2454] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e02f5bd902a34b68b2fe9efee6b32a48 in term 0.
I20260806 18:47:24.833678 22279 raft_consensus.cc:2454] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e02f5bd902a34b68b2fe9efee6b32a48 in term 0.
I20260806 18:47:24.834105 22376 leader_election.cc:302] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40020e316fb34301b76b4b51ef57ce23, e02f5bd902a34b68b2fe9efee6b32a48; no voters:
I20260806 18:47:24.834383 22584 raft_consensus.cc:2781] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:47:24.834497 22584 raft_consensus.cc:482] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:47:24.834607 22584 raft_consensus.cc:3037] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Advancing to term 1
W20260806 18:47:24.835371 22544 tablet.cc:2369] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:24.835844 22584 raft_consensus.cc:504] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.836174 22584 leader_election.cc:288] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [CANDIDATE]: Term 1 election: Requested vote from peers fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671), 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803)
I20260806 18:47:24.836535 22057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52de4ebe879443499e3a8364cba00ea5" candidate_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbf499ccd73a4dfd989d28f072aa72c3"
I20260806 18:47:24.836534 22279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52de4ebe879443499e3a8364cba00ea5" candidate_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40020e316fb34301b76b4b51ef57ce23"
I20260806 18:47:24.836643 22057 raft_consensus.cc:3037] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:24.836656 22279 raft_consensus.cc:3037] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:24.837667 22057 raft_consensus.cc:2454] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e02f5bd902a34b68b2fe9efee6b32a48 in term 1.
I20260806 18:47:24.837689 22279 raft_consensus.cc:2454] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e02f5bd902a34b68b2fe9efee6b32a48 in term 1.
I20260806 18:47:24.837942 22378 leader_election.cc:302] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e02f5bd902a34b68b2fe9efee6b32a48, fbf499ccd73a4dfd989d28f072aa72c3; no voters:
I20260806 18:47:24.838115 22584 raft_consensus.cc:2781] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:24.838382 22584 raft_consensus.cc:686] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 LEADER]: Becoming Leader. State: Replica: e02f5bd902a34b68b2fe9efee6b32a48, State: Running, Role: LEADER
I20260806 18:47:24.838685 22584 consensus_queue.cc:237] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:24.841775 21832 catalog_manager.cc:5495] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 reported cstate change: term changed from 0 to 1, leader changed from <none> to e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67). New cstate: current_term: 1 leader_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } health_report { overall_health: UNKNOWN } } }
W20260806 18:47:24.863229 22349 tablet.cc:2369] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:24.874086 22567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.870845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.1.245.67:35763)
0806 18:47:24.870982 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:24.870987 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:24.871002 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:47:24.871109 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:24.871112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:24.871167 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:24.871275 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:47:24.871519 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.872018 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.872772 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:24.872978 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:24.873042 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:24.873258 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:47:24.873709 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:47:24.873799 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:24.873844 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 18:47:24.873905 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:47:24.880450 22057 raft_consensus.cc:1261] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Refusing update from remote peer e02f5bd902a34b68b2fe9efee6b32a48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:24.880450 22279 raft_consensus.cc:1261] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 1 FOLLOWER]: Refusing update from remote peer e02f5bd902a34b68b2fe9efee6b32a48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:24.881217 22584 consensus_queue.cc:1035] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:24.881482 22592 consensus_queue.cc:1035] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:24.889581 22598 mvcc.cc:205] Tried to move back new op lower bound from 7315628215821131776 to 7315628215661621248. Current Snapshot: MvccSnapshot[applied={T|T < 7315628215821131776}]
I20260806 18:47:24.897519 21832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:47:24.897719 21832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:47:24.900270 21832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Sending DeleteTablet for 3 replicas of tablet 52de4ebe879443499e3a8364cba00ea5
I20260806 18:47:24.901183 22023 tablet_service.cc:1514] Processing DeleteTablet for tablet 52de4ebe879443499e3a8364cba00ea5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:24 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:43466
I20260806 18:47:24.901295 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 21921
I20260806 18:47:24.901901 22244 tablet_service.cc:1514] Processing DeleteTablet for tablet 52de4ebe879443499e3a8364cba00ea5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:24 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:58402
I20260806 18:47:24.908771 22447 tablet_service.cc:1514] Processing DeleteTablet for tablet 52de4ebe879443499e3a8364cba00ea5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:24 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:42970
I20260806 18:47:24.909361 22611 tablet_replica.cc:330] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: stopping tablet replica
I20260806 18:47:24.909746 22611 raft_consensus.cc:2229] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:47:24.910135 22611 raft_consensus.cc:2258] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:24.910171 22614 tablet_replica.cc:330] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: stopping tablet replica
I20260806 18:47:24.910634 22614 raft_consensus.cc:2229] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:47:24.911149 22614 raft_consensus.cc:2258] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:24.912081 22611 ts_tablet_manager.cc:1905] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:24.912755 22614 ts_tablet_manager.cc:1905] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:24.914816 22611 ts_tablet_manager.cc:1918] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:24.914918 22611 log.cc:1196] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/52de4ebe879443499e3a8364cba00ea5
I20260806 18:47:24.915239 22611 ts_tablet_manager.cc:1939] T 52de4ebe879443499e3a8364cba00ea5 P 40020e316fb34301b76b4b51ef57ce23: Deleting consensus metadata
I20260806 18:47:24.915803 22614 ts_tablet_manager.cc:1918] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:24.915975 22614 log.cc:1196] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/52de4ebe879443499e3a8364cba00ea5
I20260806 18:47:24.916361 22614 ts_tablet_manager.cc:1939] T 52de4ebe879443499e3a8364cba00ea5 P e02f5bd902a34b68b2fe9efee6b32a48: Deleting consensus metadata
I20260806 18:47:24.916548 21806 catalog_manager.cc:4841] TS 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803): tablet 52de4ebe879443499e3a8364cba00ea5 (table test-table [id=a87aa70f700c4976a11954fcc1a9d61a]) successfully deleted
W20260806 18:47:24.916561 21808 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:44671 (error 108)
I20260806 18:47:24.917524 21808 catalog_manager.cc:4841] TS e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763): tablet 52de4ebe879443499e3a8364cba00ea5 (table test-table [id=a87aa70f700c4976a11954fcc1a9d61a]) successfully deleted
I20260806 18:47:24.917654 22565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:24.917306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:44671 (local address 127.0.0.1:43476)
0806 18:47:24.917443 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:24.917515 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260806 18:47:24.917809 21808 catalog_manager.cc:4584] TS fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52de4ebe879443499e3a8364cba00ea5: Network error: Client connection negotiation failed: client connection to 127.1.245.65:44671: connect: Connection refused (error 111)
I20260806 18:47:24.917917 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 22147
I20260806 18:47:24.932241 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 22354
I20260806 18:47:24.942857 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 21783
I20260806 18:47:24.955521 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:46209
--webserver_interface=127.1.245.126
--webserver_port=46791
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:46209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:25.068782 22617 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:25.069085 22617 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:25.069144 22617 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:25.073297 22617 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:25.073369 22617 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:25.073396 22617 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:25.073416 22617 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:25.073438 22617 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:25.079007 22617 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:46209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:46209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=46791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22617
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:25.080116 22617 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:25.081166 22617 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:25.095105 22627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:25.095120 22632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:25.095105 22628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:25.095610 22617 server_base.cc:1034] running on GCE node
I20260806 18:47:25.096205 22617 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:25.097370 22617 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:25.098520 22617 hybrid_clock.cc:648] HybridClock initialized: now 1786042045098491 us; error 44 us; skew 500 ppm
Aug 06 18:47:25 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:25.101208 22617 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:47:25.102123 22617 webserver.cc:460] Webserver started at http://127.1.245.126:46791/ using document root <none> and password file <none>
I20260806 18:47:25.102736 22617 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:25.102819 22617 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:25.106715 22617 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:47:25.108866 22641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:25.110105 22617 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:47:25.110226 22617 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "fdd9089a3f964ac79b3fe24f90c57be5"
format_stamp: "Formatted at 2026-08-06 18:47:23 on dist-test-slave-fgb3"
server_key: "064b15d794c6e7ef83059d13476345f9"
server_key_iv: "827417438c1a7a96fdcb2d5f3a02a95e"
server_key_version: "encryptionkey@0"
I20260806 18:47:25.110863 22617 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:25.125468 22617 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:25.128261 22617 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:25.128453 22617 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:25.135233 22617 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:46209
I20260806 18:47:25.135486 22719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:46209 every 8 connection(s)
I20260806 18:47:25.136343 22617 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:47:25.140777 22720 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:47:25.142138 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 22617
I20260806 18:47:25.142545 22720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Bootstrap starting.
I20260806 18:47:25.142556 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:44671
--local_ip_for_outbound_sockets=127.1.245.65
--tserver_master_addrs=127.1.245.126:46209
--webserver_port=38835
--webserver_interface=127.1.245.65
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:47:25.160529 22720 log.cc:824] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:25.169445 22720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:47:25.169806 22720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Bootstrap complete.
I20260806 18:47:25.173830 22720 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:25.174235 22720 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdd9089a3f964ac79b3fe24f90c57be5, State: Initialized, Role: FOLLOWER
I20260806 18:47:25.174741 22720 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:25.174863 22720 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:47:25.174945 22720 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:47:25.175037 22720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:47:25.176935 22720 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:25.177270 22720 leader_election.cc:302] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdd9089a3f964ac79b3fe24f90c57be5; no voters:
I20260806 18:47:25.177582 22720 leader_election.cc:288] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:47:25.177695 22726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:47:25.177901 22726 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 2 LEADER]: Becoming Leader. State: Replica: fdd9089a3f964ac79b3fe24f90c57be5, State: Running, Role: LEADER
I20260806 18:47:25.178217 22726 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:25.178931 22720 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:25.179229 22726 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdd9089a3f964ac79b3fe24f90c57be5. Latest consensus state: current_term: 2 leader_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } } }
I20260806 18:47:25.179436 22726 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:25.180536 22727 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } } }
I20260806 18:47:25.180709 22727 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:25.180714 22732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:47:25.185894 22732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:47:25.186554 22732 catalog_manager.cc:1259] Loaded cluster ID: 3931e0403885482abf8dfccacfb5f62b
I20260806 18:47:25.186617 22732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:47:25.188164 22732 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 18:47:25.192543 22732 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 18:47:25.198213 22732 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:47:25.199050 22732 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Loaded TSK: 0
I20260806 18:47:25.200290 22732 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:47:25.262337 22724 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:25.262591 22724 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:25.262634 22724 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:25.266604 22724 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:25.266659 22724 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:25.266721 22724 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:47:25.272198 22724 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:44671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=38835
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22724
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:25.273222 22724 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:25.274245 22724 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:25.285193 22758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:25.285244 22724 server_base.cc:1034] running on GCE node
W20260806 18:47:25.285497 22759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:25.285504 22761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:25.286068 22724 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:25.286937 22724 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:25.288153 22724 hybrid_clock.cc:648] HybridClock initialized: now 1786042045288130 us; error 34 us; skew 500 ppm
Aug 06 18:47:25 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:25.291764 22724 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:47:25.292927 22724 webserver.cc:460] Webserver started at http://127.1.245.65:38835/ using document root <none> and password file <none>
I20260806 18:47:25.293668 22724 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:25.293787 22724 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:25.299245 22724 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:47:25.301856 22771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:25.303344 22724 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:47:25.303467 22724 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "fbf499ccd73a4dfd989d28f072aa72c3"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "48b8ca7f816276b85748ffba037bb9f2"
server_key_iv: "7de051148e1ff69d28397496e0a0fd6a"
server_key_version: "encryptionkey@0"
I20260806 18:47:25.304150 22724 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:25.333504 22724 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:25.336614 22724 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:25.336782 22724 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:25.337380 22724 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:25.338541 22783 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:47:25.343385 22724 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:47:25.343454 22724 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260806 18:47:25.343514 22724 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:47:25.345702 22724 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:47:25.345765 22724 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:47:25.346094 22783 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Bootstrap starting.
I20260806 18:47:25.357326 22724 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:44671
I20260806 18:47:25.357339 22941 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:44671 every 8 connection(s)
I20260806 18:47:25.358664 22724 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:47:25.359856 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 22724
I20260806 18:47:25.360293 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:33803
--local_ip_for_outbound_sockets=127.1.245.66
--tserver_master_addrs=127.1.245.126:46209
--webserver_port=41045
--webserver_interface=127.1.245.66
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:47:25.368896 22783 log.cc:824] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:47:25 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:25.377358 22945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.360885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:52139)
0806 18:47:25.361532 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:25.361597 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:25.362638 (+ 1041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:25.364508 (+ 1870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:25.364518 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:25.365089 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:25.365733 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.365754 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.367821 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.367825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.369124 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.369136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.369910 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.369914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.370104 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.370858 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:25.370875 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:25.372301 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:25.374364 (+ 2063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:25.374370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:25.374384 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:25.374648 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:25.374994 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:25.374997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:25.374998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:25.375129 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:25.375711 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:25.375717 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:25.375953 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 18:47:25.376109 (+ 156us) client_negotiation.cc:240] Negotiation successful
0806 18:47:25.376449 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":181,"threads_started":1}
I20260806 18:47:25.377347 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.362303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:52139 (local address 127.1.245.126:46209)
0806 18:47:25.362690 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.362709 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.362755 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:47:25.363141 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.363159 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.363624 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.363995 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:25.365975 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.367662 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.369272 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.369756 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.369923 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.372527 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:25.372532 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:25.372545 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:25.372574 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:25.374189 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.374763 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.374769 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.374775 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.374899 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.375230 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.375234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.375236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.375595 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:25.375720 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.376265 (+ 545us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.376551 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":116,"threads_started":1}
I20260806 18:47:25.377688 22783 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:47:25.378130 22783 tablet_bootstrap.cc:492] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Bootstrap complete.
I20260806 18:47:25.379732 22783 ts_tablet_manager.cc:1397] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Time spent bootstrapping tablet: real 0.034s user 0.013s sys 0.018s
I20260806 18:47:25.380030 22943 heartbeater.cc:347] Connected to a master server at 127.1.245.126:46209
I20260806 18:47:25.380282 22943 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:25.380750 22943 heartbeater.cc:511] Master 127.1.245.126:46209 requested a full tablet report, sending...
I20260806 18:47:25.383039 22664 ts_manager.cc:194] Registered new tserver with Master: fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671)
I20260806 18:47:25.382650 22783 raft_consensus.cc:348] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:25.383155 22783 raft_consensus.cc:729] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbf499ccd73a4dfd989d28f072aa72c3, State: Initialized, Role: FOLLOWER
I20260806 18:47:25.383625 22783 consensus_queue.cc:260] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } }
I20260806 18:47:25.384616 22783 ts_tablet_manager.cc:1428] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:47:25.386788 22664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:52139
I20260806 18:47:25.389488 22943 heartbeater.cc:503] Master 127.1.245.126:46209 was elected leader, sending a full tablet report...
Aug 06 18:47:25 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:47:25.395884 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.385987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:44671 (local address 127.0.0.1:43488)
0806 18:47:25.386427 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:25.386475 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:25.386592 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:25.387235 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:25.387241 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:25.387667 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:25.387796 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.387805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.388747 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.388750 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.389355 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.389364 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.390105 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.390109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.390217 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.390803 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:25.390818 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:25.392207 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:25.394135 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:25.394144 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:25.394148 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:25.394468 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:25.394758 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:25.394760 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:25.394761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:25.394832 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:25.395362 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:25.395368 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:25.395475 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 18:47:25.395594 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 18:47:25.395744 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":114,"threads_started":1}
I20260806 18:47:25.396039 22955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.386052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.1.245.65:44671)
0806 18:47:25.386765 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.386776 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.386800 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:47:25.386866 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.386883 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.387080 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.387210 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:25.387914 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.388640 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.389469 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.389887 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.389980 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.392339 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:25.392349 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:25.392361 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:25.392388 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:25.394034 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.394570 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.394576 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.394578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.394677 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.394912 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.394915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.394917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.395220 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:25.395342 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.395701 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.395882 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":88,"threads_started":1}
I20260806 18:47:25.396960 22840 tablet_service.cc:1514] Processing DeleteTablet for tablet 52de4ebe879443499e3a8364cba00ea5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:24 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:43488
I20260806 18:47:25.397974 22956 tablet_replica.cc:330] stopping tablet replica
I20260806 18:47:25.398300 22956 raft_consensus.cc:2229] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:47:25.398612 22956 raft_consensus.cc:2258] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:25.399572 22956 ts_tablet_manager.cc:1905] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:25.403123 22956 ts_tablet_manager.cc:1918] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:25.403219 22956 log.cc:1196] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/52de4ebe879443499e3a8364cba00ea5
I20260806 18:47:25.403463 22956 ts_tablet_manager.cc:1939] T 52de4ebe879443499e3a8364cba00ea5 P fbf499ccd73a4dfd989d28f072aa72c3: Deleting consensus metadata
I20260806 18:47:25.405189 22646 catalog_manager.cc:4841] TS fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671): tablet 52de4ebe879443499e3a8364cba00ea5 (table test-table [id=a87aa70f700c4976a11954fcc1a9d61a]) successfully deleted
W20260806 18:47:25.484941 22948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:25.485206 22948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:25.485252 22948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:25.489418 22948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:25.489486 22948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:25.489549 22948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:47:25.494946 22948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:33803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=41045
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:25.496057 22948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:25.497097 22948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:25.506765 22966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:25.506840 22965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:25.506830 22968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:25.507113 22948 server_base.cc:1034] running on GCE node
I20260806 18:47:25.507555 22948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:25.508170 22948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:25.509379 22948 hybrid_clock.cc:648] HybridClock initialized: now 1786042045509327 us; error 36 us; skew 500 ppm
Aug 06 18:47:25 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:25.512064 22948 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:47:25.512990 22948 webserver.cc:460] Webserver started at http://127.1.245.66:41045/ using document root <none> and password file <none>
I20260806 18:47:25.513551 22948 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:25.513640 22948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:25.517829 22948 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:47:25.519950 22982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:25.521178 22948 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:47:25.521292 22948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "40020e316fb34301b76b4b51ef57ce23"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "d6bf1e9ea29078a5b218dbd43bf93b76"
server_key_iv: "c32eb7416063d6b71505b8504ea6212c"
server_key_version: "encryptionkey@0"
I20260806 18:47:25.521842 22948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:25.560379 22948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:25.563405 22948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:25.563594 22948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:25.564225 22948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:25.565090 22948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:25.565140 22948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:25.565194 22948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:25.565223 22948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:25.573894 22948 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:33803
I20260806 18:47:25.573920 23148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:33803 every 8 connection(s)
I20260806 18:47:25.574889 22948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:47:25.577991 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 22948
I20260806 18:47:25.578380 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:35763
--local_ip_for_outbound_sockets=127.1.245.67
--tserver_master_addrs=127.1.245.126:46209
--webserver_port=41171
--webserver_interface=127.1.245.67
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:47:25 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:25.589457 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.576683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:38955 (local address 127.1.245.126:46209)
0806 18:47:25.576832 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.576836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.577657 (+ 821us) server_negotiation.cc:407] Connection header received
0806 18:47:25.579141 (+ 1484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.579145 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.579205 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.579283 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:25.580497 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.581088 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.582674 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.582941 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.583047 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.586034 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:25.586037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:25.586039 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:25.586064 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:25.587439 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.587922 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.587925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.587926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.588001 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.588324 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.588328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.588329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.588524 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:25.588610 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.589124 (+ 514us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.589267 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:47:25.590273 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.577057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:38955)
0806 18:47:25.577514 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:25.577556 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:25.578817 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:25.579462 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:25.579469 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:25.579834 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:25.580360 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.580377 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.581215 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.581218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.582541 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.582553 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.583122 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.583126 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.583325 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.583917 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:25.583938 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:25.585863 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:25.587548 (+ 1685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:25.587555 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:25.587571 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:25.587820 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:25.588084 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:25.588087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:25.588089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:25.588236 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:25.588626 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:25.588633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:25.588893 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 18:47:25.589065 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:47:25.589405 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":117,"threads_started":1}
I20260806 18:47:25.591630 23150 heartbeater.cc:347] Connected to a master server at 127.1.245.126:46209
I20260806 18:47:25.591902 23150 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:25.592420 23150 heartbeater.cc:511] Master 127.1.245.126:46209 requested a full tablet report, sending...
I20260806 18:47:25.593513 22664 ts_manager.cc:194] Registered new tserver with Master: 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803)
I20260806 18:47:25.594276 22664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:38955
W20260806 18:47:25.693037 23155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:25.693270 23155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:25.693317 23155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:25.697263 23155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:25.697315 23155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:25.697377 23155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:47:25.702488 23155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:35763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=41171
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:25.703471 23155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:25.704469 23155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:25.713824 23167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:25.713827 23163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:25.713798 23164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:25.714079 23155 server_base.cc:1034] running on GCE node
I20260806 18:47:25.714465 23155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:25.715065 23155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:25.716212 23155 hybrid_clock.cc:648] HybridClock initialized: now 1786042045716191 us; error 40 us; skew 500 ppm
Aug 06 18:47:25 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:25.718829 23155 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:47:25.719682 23155 webserver.cc:460] Webserver started at http://127.1.245.67:41171/ using document root <none> and password file <none>
I20260806 18:47:25.720198 23155 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:25.720274 23155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:25.723927 23155 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:47:25.725868 23177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:25.727072 23155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260806 18:47:25.727170 23155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "e02f5bd902a34b68b2fe9efee6b32a48"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "082e5ec642df46f17a3c471c38a2cdf3"
server_key_iv: "8722c6b2b76bd7ec716fbe4ed5034f5e"
server_key_version: "encryptionkey@0"
I20260806 18:47:25.727682 23155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:25.739684 23155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:25.742151 23155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:25.742308 23155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:25.742892 23155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:25.743677 23155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:25.743722 23155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:25.743763 23155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:25.743791 23155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:25.752071 23155 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:35763
I20260806 18:47:25.752091 23348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:35763 every 8 connection(s)
I20260806 18:47:25.753037 23155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:47:25.755551 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 23155
Aug 06 18:47:25 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042045, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:25.764516 23353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.756374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38284 (local address 127.1.245.126:46209)
0806 18:47:25.756576 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.756580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.756728 (+ 148us) server_negotiation.cc:407] Connection header received
0806 18:47:25.756877 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.756880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.756930 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.757004 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:25.757758 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.758308 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.759174 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.759416 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.759503 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.760942 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:25.760945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:25.760947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:25.760969 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:25.762426 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.763054 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.763056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.763058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.763129 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.763643 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.763646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.763647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.763833 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:25.764006 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.764209 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.764331 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":58,"threads_started":1}
I20260806 18:47:25.766085 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.754818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:45535 (local address 127.1.245.126:46209)
0806 18:47:25.754969 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.754973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.756219 (+ 1246us) server_negotiation.cc:407] Connection header received
0806 18:47:25.757117 (+ 898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.757120 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.757168 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.757240 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:25.758389 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.758867 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.759959 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.760253 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.760371 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.762835 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:25.762838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:25.762840 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:25.762863 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:25.764018 (+ 1155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.764596 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.764599 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.764601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.764691 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:25.765058 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:25.765062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:25.765063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:25.765227 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:25.765321 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.765841 (+ 520us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.765951 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:47:25.766861 23351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.755195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:45535)
0806 18:47:25.756065 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:25.756117 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:25.756919 (+ 802us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:25.757419 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:25.757425 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:25.757746 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:25.758228 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.758242 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.758986 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.758989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.759834 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.759846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.760397 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.760401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.760543 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.761017 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:25.761033 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:25.762688 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:25.764135 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:25.764141 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:25.764155 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:25.764491 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:25.764783 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:25.764786 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:25.764788 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:25.764960 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:25.765330 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:25.765334 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:25.765623 (+ 289us) client_negotiation.cc:769] Sending connection context
0806 18:47:25.765775 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 18:47:25.766023 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":533,"threads_started":1}
I20260806 18:47:25.767845 23349 heartbeater.cc:347] Connected to a master server at 127.1.245.126:46209
I20260806 18:47:25.768067 23349 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:25.768517 23349 heartbeater.cc:511] Master 127.1.245.126:46209 requested a full tablet report, sending...
I20260806 18:47:25.769430 22664 ts_manager.cc:194] Registered new tserver with Master: e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763)
I20260806 18:47:25.769953 22664 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:45535
I20260806 18:47:25.777987 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:47:25.782894 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.778679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38286 (local address 127.1.245.126:46209)
0806 18:47:25.778800 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.778805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.779021 (+ 216us) server_negotiation.cc:407] Connection header received
0806 18:47:25.779173 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.779176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.779319 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.779431 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:47:25.779659 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.780404 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.781366 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.781601 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.781698 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.782041 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:47:25.782495 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:47:25.782615 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.782691 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.782752 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:47:25.783522 22664 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:38286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:47:25.786737 22664 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:47:25.794751 22840 tablet_service.cc:1467] Processing CreateTablet for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3 (DEFAULT_TABLE table=test-table [id=eccbc0b693b94b969755262c056f21ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:25.795321 22840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:25.797966 22783 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Bootstrap starting.
I20260806 18:47:25.798365 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.793357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:33803 (local address 127.0.0.1:58418)
0806 18:47:25.793483 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:25.793497 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:25.793575 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:25.794480 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:25.794483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:25.794501 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:25.794600 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.794606 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.795629 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.795633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.796771 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.796783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.797870 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.797874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.797993 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.798042 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 18:47:25.798109 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 18:47:25.798194 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:47:25.798628 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.793545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58418 (local address 127.1.245.66:33803)
0806 18:47:25.793980 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.793989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.794008 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:47:25.794080 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.794086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.794296 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.794514 (+ 218us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:25.794689 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.795508 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.796901 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.797733 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.797845 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.798012 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.798374 (+ 362us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.798489 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":137,"threads_started":1}
I20260806 18:47:25.798924 22783 tablet_bootstrap.cc:654] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:25.800509 22783 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: No bootstrap required, opened a new log
I20260806 18:47:25.800608 22783 ts_tablet_manager.cc:1397] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 18:47:25.801066 22783 raft_consensus.cc:348] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.801195 22783 raft_consensus.cc:374] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:25.801239 22783 raft_consensus.cc:729] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbf499ccd73a4dfd989d28f072aa72c3, State: Initialized, Role: FOLLOWER
I20260806 18:47:25.801267 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.793751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35763 (local address 127.0.0.1:42992)
0806 18:47:25.794182 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:25.794194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:25.794283 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:25.796548 (+ 2265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:25.796550 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:25.796564 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:25.796633 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.796637 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.798776 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.798779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.799842 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.799854 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.800908 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.800911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.800979 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.800997 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:47:25.801063 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:47:25.801135 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":98,"threads_started":1}
I20260806 18:47:25.801363 22783 consensus_queue.cc:260] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.801635 22783 ts_tablet_manager.cc:1428] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 18:47:25.801662 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.793965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42992 (local address 127.1.245.67:35763)
0806 18:47:25.795906 (+ 1941us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.795918 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.795939 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:47:25.796010 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.796016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.796203 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.796406 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:25.796743 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.797795 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.800050 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.800644 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.800742 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.800852 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.801116 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.801550 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1831,"thread_start_us":136,"threads_started":1}
I20260806 18:47:25.801836 23047 tablet_service.cc:1467] Processing CreateTablet for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3 (DEFAULT_TABLE table=test-table [id=eccbc0b693b94b969755262c056f21ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:25.803100 23047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:25.804843 23246 tablet_service.cc:1467] Processing CreateTablet for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3 (DEFAULT_TABLE table=test-table [id=eccbc0b693b94b969755262c056f21ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:25.805995 23246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:25.811957 23367 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Bootstrap starting.
I20260806 18:47:25.813948 23369 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Bootstrap starting.
I20260806 18:47:25.814654 23367 tablet_bootstrap.cc:654] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:25.815569 23367 log.cc:824] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:25.816535 23369 tablet_bootstrap.cc:654] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:25.817361 23369 log.cc:824] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:25.817802 23367 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: No bootstrap required, opened a new log
I20260806 18:47:25.817986 23367 ts_tablet_manager.cc:1397] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 18:47:25.819372 23369 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: No bootstrap required, opened a new log
I20260806 18:47:25.819558 23369 ts_tablet_manager.cc:1397] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:47:25.821867 23367 raft_consensus.cc:348] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.822160 23367 raft_consensus.cc:374] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:25.822217 23367 raft_consensus.cc:729] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40020e316fb34301b76b4b51ef57ce23, State: Initialized, Role: FOLLOWER
I20260806 18:47:25.822715 23367 consensus_queue.cc:260] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.823246 23369 raft_consensus.cc:348] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.823450 23150 heartbeater.cc:503] Master 127.1.245.126:46209 was elected leader, sending a full tablet report...
I20260806 18:47:25.823498 23369 raft_consensus.cc:374] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:25.823552 23369 raft_consensus.cc:729] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e02f5bd902a34b68b2fe9efee6b32a48, State: Initialized, Role: FOLLOWER
I20260806 18:47:25.823838 23367 ts_tablet_manager.cc:1428] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:47:25.823958 23369 consensus_queue.cc:260] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.824555 23349 heartbeater.cc:503] Master 127.1.245.126:46209 was elected leader, sending a full tablet report...
I20260806 18:47:25.825214 23369 ts_tablet_manager.cc:1428] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260806 18:47:25.825778 23151 tablet.cc:2369] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:47:25.860004 22944 tablet.cc:2369] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:25.943324 23374 raft_consensus.cc:482] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:25.943557 23374 raft_consensus.cc:504] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.944772 23374 leader_election.cc:288] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763), fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671)
I20260806 18:47:25.949012 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.945194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:43303 (local address 127.1.245.67:35763)
0806 18:47:25.945328 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.945332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.945351 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:47:25.945513 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.945516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.945566 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.945627 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:25.946031 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.946565 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.947810 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.948419 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.948506 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.948558 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.948817 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.948912 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:47:25.949011 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.945048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35763 (local address 127.1.245.66:43303)
0806 18:47:25.945212 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:25.945264 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:25.945370 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:25.945753 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:25.945755 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:25.945776 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:25.945902 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.945909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.946665 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.946668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.947679 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.947688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.948523 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.948526 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.948644 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.948672 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:47:25.948748 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 18:47:25.948819 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":1}
I20260806 18:47:25.949700 23376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.945325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:44671 (local address 127.1.245.66:44457)
0806 18:47:25.945757 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:25.945771 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:25.945870 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:25.946168 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:25.946171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:25.946188 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:25.946300 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.946306 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.947259 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.947262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.948278 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:25.948289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.949294 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.949298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:25.949394 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.949418 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:47:25.949495 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:47:25.949563 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":20,"thread_start_us":119,"threads_started":1}
I20260806 18:47:25.949734 23377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.945574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:44457 (local address 127.1.245.65:44671)
0806 18:47:25.945817 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.945822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.945836 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:47:25.945988 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.945990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.946048 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.946142 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:25.946426 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.947112 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.948410 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.949043 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.949160 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.949325 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.949534 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.949617 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":69,"threads_started":1}
I20260806 18:47:25.949693 23280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f40beb19a3d4ce9ba164d2a9f39c8a3" candidate_uuid: "40020e316fb34301b76b4b51ef57ce23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" is_pre_election: true
I20260806 18:47:25.950035 23280 raft_consensus.cc:2454] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40020e316fb34301b76b4b51ef57ce23 in term 0.
I20260806 18:47:25.950273 22876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f40beb19a3d4ce9ba164d2a9f39c8a3" candidate_uuid: "40020e316fb34301b76b4b51ef57ce23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" is_pre_election: true
I20260806 18:47:25.950546 22876 raft_consensus.cc:2454] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40020e316fb34301b76b4b51ef57ce23 in term 0.
I20260806 18:47:25.950546 22987 leader_election.cc:302] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40020e316fb34301b76b4b51ef57ce23, e02f5bd902a34b68b2fe9efee6b32a48; no voters:
I20260806 18:47:25.950815 23374 raft_consensus.cc:2781] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:47:25.950902 23374 raft_consensus.cc:482] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:47:25.950937 23374 raft_consensus.cc:3037] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:25.951990 23374 raft_consensus.cc:504] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.952314 23374 leader_election.cc:288] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [CANDIDATE]: Term 1 election: Requested vote from peers e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763), fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671)
I20260806 18:47:25.952658 23280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f40beb19a3d4ce9ba164d2a9f39c8a3" candidate_uuid: "40020e316fb34301b76b4b51ef57ce23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e02f5bd902a34b68b2fe9efee6b32a48"
I20260806 18:47:25.952708 22876 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f40beb19a3d4ce9ba164d2a9f39c8a3" candidate_uuid: "40020e316fb34301b76b4b51ef57ce23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbf499ccd73a4dfd989d28f072aa72c3"
I20260806 18:47:25.952770 23280 raft_consensus.cc:3037] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:25.952855 22876 raft_consensus.cc:3037] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:25.953817 22876 raft_consensus.cc:2454] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40020e316fb34301b76b4b51ef57ce23 in term 1.
I20260806 18:47:25.953828 23280 raft_consensus.cc:2454] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40020e316fb34301b76b4b51ef57ce23 in term 1.
I20260806 18:47:25.954098 22987 leader_election.cc:302] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40020e316fb34301b76b4b51ef57ce23, e02f5bd902a34b68b2fe9efee6b32a48; no voters:
I20260806 18:47:25.954262 23374 raft_consensus.cc:2781] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:25.954603 23374 raft_consensus.cc:686] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 1 LEADER]: Becoming Leader. State: Replica: 40020e316fb34301b76b4b51ef57ce23, State: Running, Role: LEADER
I20260806 18:47:25.954941 23374 consensus_queue.cc:237] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:25.957825 22663 catalog_manager.cc:5495] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "40020e316fb34301b76b4b51ef57ce23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } health_report { overall_health: UNKNOWN } } }
I20260806 18:47:25.991284 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:25.987786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58424 (local address 127.1.245.66:33803)
0806 18:47:25.987922 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:25.987925 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:25.987939 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:47:25.988018 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:25.988020 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:25.988079 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:25.988169 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:47:25.988344 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.988838 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.989677 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:25.989913 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:25.989997 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:25.990439 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:47:25.990887 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:47:25.991005 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:25.991085 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 18:47:25.991145 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:47:25.997356 22876 raft_consensus.cc:1261] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Refusing update from remote peer 40020e316fb34301b76b4b51ef57ce23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:25.997359 23280 raft_consensus.cc:1261] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 FOLLOWER]: Refusing update from remote peer 40020e316fb34301b76b4b51ef57ce23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:25.998080 23374 consensus_queue.cc:1035] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:25.998294 23378 consensus_queue.cc:1035] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:26.003954 23381 mvcc.cc:205] Tried to move back new op lower bound from 7315628220397735936 to 7315628220233998336. Current Snapshot: MvccSnapshot[applied={T|T < 7315628220397735936}]
W20260806 18:47:26.004359 23350 tablet.cc:2369] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:26.004500 23382 mvcc.cc:205] Tried to move back new op lower bound from 7315628220397735936 to 7315628220233998336. Current Snapshot: MvccSnapshot[applied={T|T < 7315628220397735936}]
I20260806 18:47:26.006839 23383 mvcc.cc:205] Tried to move back new op lower bound from 7315628220397735936 to 7315628220233998336. Current Snapshot: MvccSnapshot[applied={T|T < 7315628220397735936}]
I20260806 18:47:26.014037 22663 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:38286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:47:26.014199 22663 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:38286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:47:26.016656 22663 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Sending DeleteTablet for 3 replicas of tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3
I20260806 18:47:26.017418 23047 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:26 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:58418
I20260806 18:47:26.017616 22840 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:26 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:43488
I20260806 18:47:26.017711 23246 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:26 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:42992
I20260806 18:47:26.018050 23396 tablet_replica.cc:330] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: stopping tablet replica
I20260806 18:47:26.018203 23396 raft_consensus.cc:2229] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:47:26.018266 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 22724
I20260806 18:47:26.018390 23396 raft_consensus.cc:2258] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:26.018386 23398 tablet_replica.cc:330] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: stopping tablet replica
I20260806 18:47:26.018802 23398 raft_consensus.cc:2229] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:47:26.019212 23398 raft_consensus.cc:2258] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:26.023809 23398 ts_tablet_manager.cc:1905] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:26.027508 23398 ts_tablet_manager.cc:1918] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:26.027684 23398 log.cc:1196] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/4f40beb19a3d4ce9ba164d2a9f39c8a3
I20260806 18:47:26.028141 23398 ts_tablet_manager.cc:1939] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P e02f5bd902a34b68b2fe9efee6b32a48: Deleting consensus metadata
I20260806 18:47:26.029625 22646 catalog_manager.cc:4841] TS e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763): tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3 (table test-table [id=eccbc0b693b94b969755262c056f21ae]) successfully deleted
I20260806 18:47:26.034713 23395 tablet_replica.cc:330] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: stopping tablet replica
I20260806 18:47:26.035105 23395 raft_consensus.cc:2229] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:47:26.035553 23395 raft_consensus.cc:2258] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:26.037075 23395 ts_tablet_manager.cc:1905] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 18:47:26.039265 22646 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:44671 (error 108)
I20260806 18:47:26.039539 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 22948
I20260806 18:47:26.045938 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.045532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:44671 (local address 127.0.0.1:43502)
0806 18:47:26.045684 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.045798 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260806 18:47:26.049444 22646 catalog_manager.cc:4584] TS fbf499ccd73a4dfd989d28f072aa72c3 (127.1.245.65:44671): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f40beb19a3d4ce9ba164d2a9f39c8a3: Network error: Client connection negotiation failed: client connection to 127.1.245.65:44671: connect: Connection refused (error 111)
I20260806 18:47:26.055588 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 23155
I20260806 18:47:26.055768 23362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.055481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:33803 (local address 127.0.0.1:58426)
0806 18:47:26.055608 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.055670 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.66:33803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:47:26.069927 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 22617
I20260806 18:47:26.081348 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:46209
--webserver_interface=127.1.245.126
--webserver_port=46791
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:46209
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:26.191808 23400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:26.192085 23400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:26.192142 23400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:26.196174 23400 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:26.196240 23400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:26.196267 23400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:26.196296 23400 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:26.196319 23400 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:26.201640 23400 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:46209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:46209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=46791
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:26.202793 23400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:26.203830 23400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:26.212817 23409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:26.212853 23400 server_base.cc:1034] running on GCE node
W20260806 18:47:26.212960 23408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:26.213131 23412 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:26.213716 23400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:26.214912 23400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:26.216087 23400 hybrid_clock.cc:648] HybridClock initialized: now 1786042046216073 us; error 24 us; skew 500 ppm
Aug 06 18:47:26 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:26.218850 23400 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:47:26.219717 23400 webserver.cc:460] Webserver started at http://127.1.245.126:46791/ using document root <none> and password file <none>
I20260806 18:47:26.220237 23400 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:26.220294 23400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:26.223980 23400 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:47:26.226032 23426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:26.227315 23400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:47:26.227454 23400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "fdd9089a3f964ac79b3fe24f90c57be5"
format_stamp: "Formatted at 2026-08-06 18:47:23 on dist-test-slave-fgb3"
server_key: "064b15d794c6e7ef83059d13476345f9"
server_key_iv: "827417438c1a7a96fdcb2d5f3a02a95e"
server_key_version: "encryptionkey@0"
I20260806 18:47:26.227980 23400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:26.246860 23400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:26.249994 23400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:26.250164 23400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:26.257236 23400 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:46209
I20260806 18:47:26.257561 23497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:46209 every 8 connection(s)
I20260806 18:47:26.258340 23400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:47:26.262775 23499 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:47:26.264428 23499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Bootstrap starting.
I20260806 18:47:26.267941 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 23400
I20260806 18:47:26.268433 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:44671
--local_ip_for_outbound_sockets=127.1.245.65
--tserver_master_addrs=127.1.245.126:46209
--webserver_port=38835
--webserver_interface=127.1.245.65
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:47:26.283800 23499 log.cc:824] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:26.296460 23499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:47:26.296876 23499 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5: Bootstrap complete.
I20260806 18:47:26.300990 23499 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:26.301394 23499 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdd9089a3f964ac79b3fe24f90c57be5, State: Initialized, Role: FOLLOWER
I20260806 18:47:26.301878 23499 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:26.302018 23499 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:47:26.302093 23499 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:47:26.302191 23499 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:47:26.304220 23499 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:26.304548 23499 leader_election.cc:302] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdd9089a3f964ac79b3fe24f90c57be5; no voters:
I20260806 18:47:26.304869 23499 leader_election.cc:288] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:47:26.304986 23504 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:47:26.305220 23504 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [term 3 LEADER]: Becoming Leader. State: Replica: fdd9089a3f964ac79b3fe24f90c57be5, State: Running, Role: LEADER
I20260806 18:47:26.305538 23504 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } }
I20260806 18:47:26.306227 23499 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:26.306571 23504 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdd9089a3f964ac79b3fe24f90c57be5. Latest consensus state: current_term: 3 leader_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } } }
I20260806 18:47:26.306690 23504 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:26.307379 23505 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdd9089a3f964ac79b3fe24f90c57be5" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 46209 } } }
I20260806 18:47:26.308161 23505 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdd9089a3f964ac79b3fe24f90c57be5 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:26.307930 23509 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:47:26.313495 23509 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:47:26.314194 23509 catalog_manager.cc:1259] Loaded cluster ID: 3931e0403885482abf8dfccacfb5f62b
I20260806 18:47:26.314260 23509 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 18:47:26.315376 23509 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe10bee69ad google::LogMessage::Fail() at ??:0
@ 0x7fe10bee988a google::LogMessage::SendToLog() at ??:0
@ 0x7fe10bee64de google::LogMessage::Flush() at ??:0
@ 0x7fe10bee7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe116825fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe116826d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe116823579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe116867640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe10e1383d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe10e138c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe10e13a3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe10e129512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe10f76d6db start_thread at ??:0
@ 0x7fe10a9e461f clone at ??:0
W20260806 18:47:26.405736 23502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:26.405987 23502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:26.406032 23502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:26.410014 23502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:26.410066 23502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:26.410123 23502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:47:26.415319 23502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:44671
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=38835
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:26.416344 23502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:26.417358 23502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:26.427011 23567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:26.427040 23569 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:26.427224 23502 server_base.cc:1034] running on GCE node
W20260806 18:47:26.427320 23573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:26.427721 23502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:26.428331 23502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:26.429507 23502 hybrid_clock.cc:648] HybridClock initialized: now 1786042046429481 us; error 36 us; skew 500 ppm
Aug 06 18:47:26 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:26.432317 23502 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:47:26.433236 23502 webserver.cc:460] Webserver started at http://127.1.245.65:38835/ using document root <none> and password file <none>
I20260806 18:47:26.433797 23502 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:26.433863 23502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:26.437615 23502 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:47:26.439743 23583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:26.440954 23502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:47:26.441051 23502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "fbf499ccd73a4dfd989d28f072aa72c3"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "48b8ca7f816276b85748ffba037bb9f2"
server_key_iv: "7de051148e1ff69d28397496e0a0fd6a"
server_key_version: "encryptionkey@0"
I20260806 18:47:26.441577 23502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:26.452219 23502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:26.455346 23502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:26.455542 23502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:26.456183 23502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:26.457336 23601 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:47:26.462935 23502 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:47:26.463001 23502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 18:47:26.463068 23502 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:47:26.464735 23502 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:47:26.464789 23502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:47:26.465124 23601 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Bootstrap starting.
I20260806 18:47:26.474876 23502 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:44671
I20260806 18:47:26.475124 23764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:44671 every 8 connection(s)
I20260806 18:47:26.475908 23502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:47:26.486029 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 23502
I20260806 18:47:26.486485 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:33803
--local_ip_for_outbound_sockets=127.1.245.66
--tserver_master_addrs=127.1.245.126:46209
--webserver_port=41045
--webserver_interface=127.1.245.66
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:47:26.487504 23601 log.cc:824] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:26.494817 23601 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:47:26.495152 23601 tablet_bootstrap.cc:492] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Bootstrap complete.
I20260806 18:47:26.496170 23601 ts_tablet_manager.cc:1397] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Time spent bootstrapping tablet: real 0.031s user 0.025s sys 0.004s
I20260806 18:47:26.498554 23601 raft_consensus.cc:348] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:26.498966 23601 raft_consensus.cc:729] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbf499ccd73a4dfd989d28f072aa72c3, State: Initialized, Role: FOLLOWER
I20260806 18:47:26.499430 23601 consensus_queue.cc:260] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:26.500252 23601 ts_tablet_manager.cc:1428] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260806 18:47:26.545743 23769 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:47:26.477821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:41263)
0806 18:47:26.478368 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.478423 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:26.479226 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:26.544936 (+ 65710us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":140,"threads_started":1}
W20260806 18:47:26.546368 23765 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:46209 (0 consecutive failures): Network error: Failed to ping master at 127.1.245.126:46209: Client connection negotiation failed: client connection to 127.1.245.126:46209: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:47:26.547122 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.546865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:33631)
0806 18:47:26.546991 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.547061 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 18:47:26.547807 23769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.547626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:44507)
0806 18:47:26.547716 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.547757 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:47:26.547993 23765 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:47:26.616318 23770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:26.616575 23770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:26.616617 23770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:26.620635 23770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:26.620692 23770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:26.620756 23770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:47:26.625870 23770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:33803
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=41045
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:26.626863 23770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:26.627794 23770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:26.637138 23784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:26.637189 23786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:26.637270 23783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:26.637411 23770 server_base.cc:1034] running on GCE node
I20260806 18:47:26.637799 23770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:26.638523 23770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:26.639698 23770 hybrid_clock.cc:648] HybridClock initialized: now 1786042046639668 us; error 42 us; skew 500 ppm
Aug 06 18:47:26 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:26.642750 23770 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:47:26.643613 23770 webserver.cc:460] Webserver started at http://127.1.245.66:41045/ using document root <none> and password file <none>
I20260806 18:47:26.644181 23770 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:26.644256 23770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:26.648141 23770 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260806 18:47:26.650487 23798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:26.651656 23770 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:47:26.651760 23770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "40020e316fb34301b76b4b51ef57ce23"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "d6bf1e9ea29078a5b218dbd43bf93b76"
server_key_iv: "c32eb7416063d6b71505b8504ea6212c"
server_key_version: "encryptionkey@0"
I20260806 18:47:26.652281 23770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:26.668903 23770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:26.671941 23770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:26.672114 23770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:26.672693 23770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:26.673755 23808 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260806 18:47:26.677541 23808 ts_tablet_manager.cc:1852] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 18:47:26.677923 23808 ts_tablet_manager.cc:1905] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:26.681715 23808 ts_tablet_manager.cc:1918] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:26.681819 23808 log.cc:1196] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/4f40beb19a3d4ce9ba164d2a9f39c8a3
I20260806 18:47:26.682121 23808 ts_tablet_manager.cc:1939] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P 40020e316fb34301b76b4b51ef57ce23: Deleting consensus metadata
I20260806 18:47:26.682435 23770 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 18:47:26.682495 23770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.009s user 0.000s sys 0.000s
I20260806 18:47:26.682564 23770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:26.682595 23770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:26.691200 23770 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:33803
I20260806 18:47:26.691311 23982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:33803 every 8 connection(s)
I20260806 18:47:26.692071 23770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:47:26.694557 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 23770
I20260806 18:47:26.694906 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:35763
--local_ip_for_outbound_sockets=127.1.245.67
--tserver_master_addrs=127.1.245.126:46209
--webserver_port=41171
--webserver_interface=127.1.245.67
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:47:26.695502 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.694105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:47409)
0806 18:47:26.694617 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.694750 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":150,"threads_started":1}
W20260806 18:47:26.695895 23983 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:46209 (0 consecutive failures): Network error: Failed to ping master at 127.1.245.126:46209: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
I20260806 18:47:26.696568 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.696308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:60973)
0806 18:47:26.696436 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.696503 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260806 18:47:26.697172 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.696973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:34067)
0806 18:47:26.697067 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.697119 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:47:26.697330 23983 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:47:26.807054 23988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:26.807310 23988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:26.807351 23988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:26.811324 23988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:26.811371 23988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:26.811432 23988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:47:26.816479 23988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:35763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=41171
--tserver_master_addrs=127.1.245.126:46209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:26.817458 23988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:26.818451 23988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:26.827687 24002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:26.827791 23988 server_base.cc:1034] running on GCE node
W20260806 18:47:26.827759 24005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:26.827685 23999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:26.828270 23988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:26.828848 23988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:26.830001 23988 hybrid_clock.cc:648] HybridClock initialized: now 1786042046829980 us; error 39 us; skew 500 ppm
Aug 06 18:47:26 dist-test-slave-fgb3 krb5kdc[21764](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:26.832544 23988 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:47:26.833341 23988 webserver.cc:460] Webserver started at http://127.1.245.67:41171/ using document root <none> and password file <none>
I20260806 18:47:26.833889 23988 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:26.833953 23988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:26.837551 23988 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:47:26.839504 24019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:26.840653 23988 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:47:26.840757 23988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "e02f5bd902a34b68b2fe9efee6b32a48"
format_stamp: "Formatted at 2026-08-06 18:47:24 on dist-test-slave-fgb3"
server_key: "082e5ec642df46f17a3c471c38a2cdf3"
server_key_iv: "8722c6b2b76bd7ec716fbe4ed5034f5e"
server_key_version: "encryptionkey@0"
I20260806 18:47:26.841260 23988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:26.857580 23988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:26.860388 23988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:26.860558 23988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:26.861152 23988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:26.861974 23988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:26.862021 23988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:26.862073 23988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:26.862102 23988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:26.870440 24198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:35763 every 8 connection(s)
I20260806 18:47:26.870443 23988 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:35763
I20260806 18:47:26.871425 23988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:47:26.872121 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 23988
I20260806 18:47:26.875180 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.873556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:45891)
0806 18:47:26.874017 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.874483 (+ 466us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":114,"threads_started":1}
W20260806 18:47:26.875633 24199 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:46209 (0 consecutive failures): Network error: Failed to ping master at 127.1.245.126:46209: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
I20260806 18:47:26.876161 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.875965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:37193)
0806 18:47:26.876061 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.876106 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 18:47:26.876724 24203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:26.876535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:50563)
0806 18:47:26.876631 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:26.876675 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:47:26.876869 24199 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:47:27.548970 24204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:27.548559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:43041)
0806 18:47:27.548793 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:27.548848 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":71,"threads_started":1}
I20260806 18:47:27.698479 24205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:27.697980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:39045)
0806 18:47:27.698265 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:27.698365 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":77,"threads_started":1}
I20260806 18:47:27.761842 24206 raft_consensus.cc:482] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:27.762058 24206 raft_consensus.cc:504] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:27.763278 24206 leader_election.cc:288] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803), e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763)
Aug 06 18:47:27 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.66@KRBTEST.COM
Aug 06 18:47:27 dist-test-slave-fgb3 krb5kdc[21764](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042046, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.67@KRBTEST.COM
I20260806 18:47:27.780154 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:27.763565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:33803 (local address 127.1.245.65:57381)
0806 18:47:27.764382 (+ 817us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:27.764393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:27.764483 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:27.766236 (+ 1753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:27.766243 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:27.766849 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:27.767667 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:27.767688 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:27.769710 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:27.769714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:27.771270 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:27.771283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:27.772282 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:27.772286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:27.772551 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:27.773305 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:27.773335 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:27.775460 (+ 2125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:27.777536 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:27.777543 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:27.777558 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:27.777912 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:27.778299 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:27.778302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:27.778304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:27.778530 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:27.779182 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:27.779195 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:27.779425 (+ 230us) client_negotiation.cc:769] Sending connection context
0806 18:47:27.779585 (+ 160us) client_negotiation.cc:240] Negotiation successful
0806 18:47:27.779954 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"mutex_wait_us":456,"spinlock_wait_cycles":9088,"thread_start_us":81,"threads_started":1}
I20260806 18:47:27.780170 24209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:27.763951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:49745 (local address 127.1.245.67:35763)
0806 18:47:27.764417 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:27.764425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:27.764462 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:47:27.765318 (+ 856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:27.765327 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:27.765787 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:27.766130 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:27.767932 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:27.769529 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:27.771421 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:27.772159 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:27.772335 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:27.775875 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:27.775881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:27.775892 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:27.775939 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:27.777721 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:27.778194 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:27.778199 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:27.778202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:27.778373 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:27.778637 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:27.778640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:27.778641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:27.779062 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:27.779178 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:27.779738 (+ 560us) server_negotiation.cc:299] Negotiation successful
0806 18:47:27.780010 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":137,"threads_started":1}
I20260806 18:47:27.780189 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:27.763832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:57381 (local address 127.1.245.66:33803)
0806 18:47:27.764287 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:27.764303 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:27.764480 (+ 177us) server_negotiation.cc:407] Connection header received
0806 18:47:27.765366 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:27.765384 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:27.765786 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:27.766130 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:27.767931 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:27.769529 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:27.771438 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:27.772159 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:27.772336 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:27.775713 (+ 3377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:27.775722 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:27.775739 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:27.775778 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:27.777404 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:27.778041 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:27.778047 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:27.778053 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:27.778208 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:27.778629 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:27.778632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:27.778634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:27.779062 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:27.779200 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:27.779734 (+ 534us) server_negotiation.cc:299] Negotiation successful
0806 18:47:27.779993 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":100,"threads_started":1}
I20260806 18:47:27.780182 24204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:27.763565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35763 (local address 127.1.245.65:49745)
0806 18:47:27.764074 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:27.764090 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:27.764199 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:27.766236 (+ 2037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:27.766243 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:27.766849 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:27.767664 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:27.767688 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:27.769710 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:27.769714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:27.771270 (+ 1556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:27.771283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:27.772282 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:27.772286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:27.772555 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:27.773313 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:27.773341 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:27.775693 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:27.777834 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:27.777837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:27.777839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:27.778080 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:27.778463 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:27.778465 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:27.778467 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:27.778545 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:27.779180 (+ 635us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:27.779187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:27.779424 (+ 237us) client_negotiation.cc:769] Sending connection context
0806 18:47:27.779585 (+ 161us) client_negotiation.cc:240] Negotiation successful
0806 18:47:27.780033 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":456,"spinlock_wait_cycles":12160}
W20260806 18:47:27.783205 23589 leader_election.cc:341] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763): Not found: Tablet not found: 4f40beb19a3d4ce9ba164d2a9f39c8a3
W20260806 18:47:27.783358 23586 leader_election.cc:341] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803): Not found: Tablet not found: 4f40beb19a3d4ce9ba164d2a9f39c8a3
I20260806 18:47:27.783437 23586 leader_election.cc:302] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbf499ccd73a4dfd989d28f072aa72c3; no voters: 40020e316fb34301b76b4b51ef57ce23, e02f5bd902a34b68b2fe9efee6b32a48
I20260806 18:47:27.783707 24206 raft_consensus.cc:2726] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:47:27.877771 24211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:27.877364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:38675)
0806 18:47:27.877621 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:27.877673 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":90,"threads_started":1}
I20260806 18:47:28.551527 24212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:28.550999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:41459)
0806 18:47:28.551331 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:28.551406 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":60,"threads_started":1}
I20260806 18:47:28.699661 24213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:28.699221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:49957)
0806 18:47:28.699494 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:28.699556 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":82,"threads_started":1}
I20260806 18:47:28.879125 24214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:28.878631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:33905)
0806 18:47:28.878906 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:28.878981 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":90,"threads_started":1}
I20260806 18:47:29.552731 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:29.552285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:36933)
0806 18:47:29.552568 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:29.552631 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260806 18:47:29.700778 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:29.700386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:47213)
0806 18:47:29.700629 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:29.700682 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260806 18:47:29.880332 24219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:29.879889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:53899)
0806 18:47:29.880173 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:29.880235 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260806 18:47:30.011490 24221 raft_consensus.cc:482] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:30.011649 24221 raft_consensus.cc:504] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:30.012105 24221 leader_election.cc:288] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803), e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763)
W20260806 18:47:30.012976 23589 leader_election.cc:341] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763): Not found: Tablet not found: 4f40beb19a3d4ce9ba164d2a9f39c8a3
W20260806 18:47:30.013095 23586 leader_election.cc:341] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803): Not found: Tablet not found: 4f40beb19a3d4ce9ba164d2a9f39c8a3
I20260806 18:47:30.013165 23586 leader_election.cc:302] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbf499ccd73a4dfd989d28f072aa72c3; no voters: 40020e316fb34301b76b4b51ef57ce23, e02f5bd902a34b68b2fe9efee6b32a48
I20260806 18:47:30.013300 24221 raft_consensus.cc:2726] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:47:30.553931 24224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:30.553501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:57323)
0806 18:47:30.553769 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:30.553829 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260806 18:47:30.702107 24225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:30.701591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:46161)
0806 18:47:30.701904 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:30.701980 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":80,"threads_started":1}
I20260806 18:47:30.881536 24226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:30.881092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:47579)
0806 18:47:30.881376 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:30.881437 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":132,"threads_started":1}
I20260806 18:47:31.555362 24228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:31.554852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:33403)
0806 18:47:31.555202 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:31.555263 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":95,"threads_started":1}
I20260806 18:47:31.703709 24229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:31.703130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:46283)
0806 18:47:31.703480 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:31.703561 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":101,"threads_started":1}
I20260806 18:47:31.882980 24231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:31.882520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:49747)
0806 18:47:31.882812 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:31.882876 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":101,"threads_started":1}
I20260806 18:47:32.506851 24232 raft_consensus.cc:482] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:32.507001 24232 raft_consensus.cc:504] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:32.507388 24232 leader_election.cc:288] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803), e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763)
W20260806 18:47:32.508251 23586 leader_election.cc:341] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803): Not found: Tablet not found: 4f40beb19a3d4ce9ba164d2a9f39c8a3
W20260806 18:47:32.508397 23589 leader_election.cc:341] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763): Not found: Tablet not found: 4f40beb19a3d4ce9ba164d2a9f39c8a3
I20260806 18:47:32.508481 23589 leader_election.cc:302] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbf499ccd73a4dfd989d28f072aa72c3; no voters: 40020e316fb34301b76b4b51ef57ce23, e02f5bd902a34b68b2fe9efee6b32a48
I20260806 18:47:32.508623 24232 raft_consensus.cc:2726] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:47:32.556619 24233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:32.556161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:59687)
0806 18:47:32.556465 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:32.556520 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":94,"threads_started":1}
I20260806 18:47:32.704952 24237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:32.704508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:38827)
0806 18:47:32.704775 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:32.704845 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":99,"threads_started":1}
I20260806 18:47:32.884208 24238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:32.883754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:44687)
0806 18:47:32.884035 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:32.884100 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":79,"threads_started":1}
I20260806 18:47:33.557904 24240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:33.557459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:35083)
0806 18:47:33.557736 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:33.557799 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":88,"threads_started":1}
I20260806 18:47:33.706226 24241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:33.705713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:47493)
0806 18:47:33.706068 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:33.706126 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":78,"threads_started":1}
I20260806 18:47:33.885690 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:33.885050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:55873)
0806 18:47:33.885513 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:33.885582 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":143,"threads_started":1}
I20260806 18:47:34.559253 24248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:34.558761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:43509)
0806 18:47:34.559086 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:34.559152 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":119,"threads_started":1}
I20260806 18:47:34.707472 24249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:34.707023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:34073)
0806 18:47:34.707297 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:34.707365 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":96,"threads_started":1}
I20260806 18:47:34.887084 24250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:34.886596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:49867)
0806 18:47:34.886912 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:34.886979 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":101,"threads_started":1}
I20260806 18:47:35.560542 24251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:35.560062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:50151)
0806 18:47:35.560372 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:35.560438 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":82,"threads_started":1}
I20260806 18:47:35.708778 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:35.708266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:32967)
0806 18:47:35.708579 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:35.708643 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":130,"threads_started":1}
I20260806 18:47:35.888513 24255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:35.888008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:43819)
0806 18:47:35.888329 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:35.888401 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
I20260806 18:47:36.562167 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:36.561573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:52253)
0806 18:47:36.561935 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:36.562026 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":120,"threads_started":1}
I20260806 18:47:36.710039 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:36.709539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:54547)
0806 18:47:36.709851 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:36.709935 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":77,"threads_started":1}
I20260806 18:47:36.883373 24259 raft_consensus.cc:482] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:36.883528 24259 raft_consensus.cc:504] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40020e316fb34301b76b4b51ef57ce23" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 33803 } } peers { permanent_uuid: "e02f5bd902a34b68b2fe9efee6b32a48" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35763 } } peers { permanent_uuid: "fbf499ccd73a4dfd989d28f072aa72c3" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44671 } }
I20260806 18:47:36.884023 24259 leader_election.cc:288] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803), e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763)
W20260806 18:47:36.884836 23586 leader_election.cc:341] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 40020e316fb34301b76b4b51ef57ce23 (127.1.245.66:33803): Not found: Tablet not found: 4f40beb19a3d4ce9ba164d2a9f39c8a3
W20260806 18:47:36.884956 23589 leader_election.cc:341] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e02f5bd902a34b68b2fe9efee6b32a48 (127.1.245.67:35763): Not found: Tablet not found: 4f40beb19a3d4ce9ba164d2a9f39c8a3
I20260806 18:47:36.885018 23589 leader_election.cc:302] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fbf499ccd73a4dfd989d28f072aa72c3; no voters: 40020e316fb34301b76b4b51ef57ce23, e02f5bd902a34b68b2fe9efee6b32a48
I20260806 18:47:36.885149 24259 raft_consensus.cc:2726] T 4f40beb19a3d4ce9ba164d2a9f39c8a3 P fbf499ccd73a4dfd989d28f072aa72c3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:47:36.889806 24260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:36.889338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:47097)
0806 18:47:36.889625 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:36.889690 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":81,"threads_started":1}
I20260806 18:47:37.563571 24261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:37.563062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:46503)
0806 18:47:37.563400 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:37.563467 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":83,"threads_started":1}
I20260806 18:47:37.711298 24262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:37.710789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:49079)
0806 18:47:37.711108 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:37.711182 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
I20260806 18:47:37.891131 24263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:37.890616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:58385)
0806 18:47:37.890950 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:37.891024 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":90,"threads_started":1}
I20260806 18:47:38.564853 24265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:38.564397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:41493)
0806 18:47:38.564677 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:38.564740 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
I20260806 18:47:38.712659 24273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:38.712189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:38365)
0806 18:47:38.712478 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:38.712548 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":87,"threads_started":1}
I20260806 18:47:38.892472 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:38.891990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:45065)
0806 18:47:38.892301 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:38.892367 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":85,"threads_started":1}
I20260806 18:47:39.566246 24276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:39.565722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:49285)
0806 18:47:39.566069 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:39.566137 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":108,"threads_started":1}
I20260806 18:47:39.714002 24279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:39.713461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:35511)
0806 18:47:39.713802 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:39.713879 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":100,"threads_started":1}
I20260806 18:47:39.893899 24280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:39.893362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:48039)
0806 18:47:39.893700 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:39.893779 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":90,"threads_started":1}
I20260806 18:47:40.567655 24281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:40.567147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:42571)
0806 18:47:40.567482 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:40.567552 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":97,"threads_started":1}
I20260806 18:47:40.715260 24282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:40.714827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:32875)
0806 18:47:40.715100 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:40.715160 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":86,"threads_started":1}
I20260806 18:47:40.895231 24284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:40.894750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:44297)
0806 18:47:40.895057 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:40.895128 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":81,"threads_started":1}
I20260806 18:47:41.569417 24285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:41.568903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.65:44883)
0806 18:47:41.569196 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:41.569291 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":80,"threads_started":1}
I20260806 18:47:41.716596 24286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:41.716171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.66:57393)
0806 18:47:41.716439 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:41.716497 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
I20260806 18:47:41.875919 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 23502
I20260806 18:47:41.887684 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 23770
I20260806 18:47:41.896679 24287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:41.896210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:46209 (local address 127.1.245.67:49441)
0806 18:47:41.896519 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:41.896578 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:46209: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":111,"threads_started":1}
I20260806 18:47:41.898291 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 23988
2026-08-06T18:47:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19483 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:47:41 dist-test-slave-fgb3 krb5kdc[24293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:47:41 dist-test-slave-fgb3 krb5kdc[24293](info): set up 2 sockets
Aug 06 18:47:41 dist-test-slave-fgb3 krb5kdc[24293](info): commencing operation
krb5kdc: starting...
W20260806 18:47:43.023438 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:47:43 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:47:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:47:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:47:43.148918 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33973
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:33973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:43.261134 24310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:43.261409 24310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:43.261451 24310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:43.265350 24310 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:47:43.265396 24310 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:47:43.265410 24310 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:47:43.265422 24310 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:47:43.265439 24310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:43.265455 24310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:43.265470 24310 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:47:43.265491 24310 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:47:43.270730 24310 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:33973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:43.271746 24310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:43.272744 24310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:47:43.281710 24310 server_base.cc:1034] running on GCE node
W20260806 18:47:43.281654 24316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:43.281654 24318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:43.281649 24321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:43.282493 24310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:43.283573 24310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:43.284818 24310 hybrid_clock.cc:648] HybridClock initialized: now 1786042063284792 us; error 33 us; skew 500 ppm
Aug 06 18:47:43 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042063, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:43.287710 24310 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:47:43.288671 24310 webserver.cc:460] Webserver started at http://127.1.245.126:43973/ using document root <none> and password file <none>
I20260806 18:47:43.289229 24310 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:43.289294 24310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:43.289487 24310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:43.291563 24310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "3ac19ea0977f4861bf1e52218e8943a7"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "b77c7ccc2c090aa7e6de6183d86adc6a"
server_key_iv: "360c6fc625ece6a29c60d426b44119b2"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.292125 24310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "3ac19ea0977f4861bf1e52218e8943a7"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "b77c7ccc2c090aa7e6de6183d86adc6a"
server_key_iv: "360c6fc625ece6a29c60d426b44119b2"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.295871 24310 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:47:43.298305 24332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:43.299577 24310 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:47:43.299700 24310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "3ac19ea0977f4861bf1e52218e8943a7"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "b77c7ccc2c090aa7e6de6183d86adc6a"
server_key_iv: "360c6fc625ece6a29c60d426b44119b2"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.299798 24310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:43.322551 24310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:43.325356 24310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:43.325543 24310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:43.332597 24310 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:33973
I20260806 18:47:43.332887 24414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:33973 every 8 connection(s)
I20260806 18:47:43.333600 24310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:47:43.335809 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 24310
I20260806 18:47:43.335947 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:47:43.336265 2005 external_mini_cluster.cc:1428] Setting key 9d5656e60623208dccf44ba9f240f640
I20260806 18:47:43.336903 24416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:43.343152 24416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Bootstrap starting.
Aug 06 18:47:43 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:43.345644 24416 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:43.346527 24416 log.cc:824] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:43.348505 24416 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: No bootstrap required, opened a new log
I20260806 18:47:43.350200 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:43.337805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49152 (local address 127.1.245.126:33973)
0806 18:47:43.338299 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:43.338347 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:43.338391 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:47:43.339098 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:43.339122 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:43.339469 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:43.339786 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:43.340612 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.341623 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.342514 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.342988 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.343137 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:43.345691 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:43.345698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:43.345715 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:43.345751 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:43.347538 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:43.347942 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:43.347947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:43.347951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:43.348062 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:43.348327 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:43.348331 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:43.348332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:43.348755 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:43.348932 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:43.349224 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:47:43.349511 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":178,"threads_started":1}
I20260806 18:47:43.352010 24416 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:47:43.352260 24416 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:43.352325 24416 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac19ea0977f4861bf1e52218e8943a7, State: Initialized, Role: FOLLOWER
I20260806 18:47:43.352762 24416 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:47:43.352892 24416 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:47:43.352938 24416 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:47:43.353018 24416 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:43.353889 24416 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:47:43.354178 24416 leader_election.cc:302] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ac19ea0977f4861bf1e52218e8943a7; no voters:
I20260806 18:47:43.354478 24416 leader_election.cc:288] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:47:43.354586 24424 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:43.354805 24424 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 1 LEADER]: Becoming Leader. State: Replica: 3ac19ea0977f4861bf1e52218e8943a7, State: Running, Role: LEADER
I20260806 18:47:43.355168 24424 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:47:43.355616 24416 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:47:43.356688 24426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ac19ea0977f4861bf1e52218e8943a7. Latest consensus state: current_term: 1 leader_uuid: "3ac19ea0977f4861bf1e52218e8943a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } } }
I20260806 18:47:43.356820 24426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:43.357112 24425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ac19ea0977f4861bf1e52218e8943a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } } }
I20260806 18:47:43.357199 24425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:47:43.357205 24436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:47:43.360060 24436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:47:43.365542 24436 catalog_manager.cc:1347] Generated new cluster ID: ec4094cab337420896a619c4b681e44c
I20260806 18:47:43.365609 24436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:47:43.399809 24436 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:47:43.400755 24436 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:47:43.409226 24436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Generated new TSK 0
I20260806 18:47:43.409896 24436 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:47:43.439167 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33973
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:43.551573 24455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:43.551841 24455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:43.551899 24455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:43.555938 24455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:43.555999 24455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:43.556080 24455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:47:43.561410 24455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24455
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:43.562505 24455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:43.563541 24455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:43.573159 24464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:43.573175 24462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:43.573184 24461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:43.573457 24455 server_base.cc:1034] running on GCE node
I20260806 18:47:43.573940 24455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:43.574615 24455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:43.575783 24455 hybrid_clock.cc:648] HybridClock initialized: now 1786042063575753 us; error 42 us; skew 500 ppm
Aug 06 18:47:43 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042063, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:43.578538 24455 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:47:43.579453 24455 webserver.cc:460] Webserver started at http://127.1.245.65:41373/ using document root <none> and password file <none>
I20260806 18:47:43.580024 24455 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:43.580090 24455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:43.580303 24455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:43.582499 24455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "284aa6ffc0eb445a9ae22629f99238d8"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "e5b9964aa3e029703c981b9f275aa96a"
server_key_iv: "26e8e9edd21e9f59eb2b2d9c0c0094f9"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.583079 24455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "284aa6ffc0eb445a9ae22629f99238d8"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "e5b9964aa3e029703c981b9f275aa96a"
server_key_iv: "26e8e9edd21e9f59eb2b2d9c0c0094f9"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.587172 24455 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:47:43.589797 24475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:43.591166 24455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:47:43.591296 24455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "284aa6ffc0eb445a9ae22629f99238d8"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "e5b9964aa3e029703c981b9f275aa96a"
server_key_iv: "26e8e9edd21e9f59eb2b2d9c0c0094f9"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.591411 24455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:43.627987 24455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:43.630940 24455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:43.631131 24455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:43.631752 24455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:43.632623 24455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:43.632673 24455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:43.632728 24455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:43.632756 24455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:43.641412 24455 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:34507
I20260806 18:47:43.641435 24648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:34507 every 8 connection(s)
I20260806 18:47:43.642271 24455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:47:43.646243 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 24455
I20260806 18:47:43.646394 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:47:43.646710 2005 external_mini_cluster.cc:1428] Setting key cf93bc6089ca035a16b231b50d708340
Aug 06 18:47:43 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042063, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:43.655691 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:43.644062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:60405 (local address 127.1.245.126:33973)
0806 18:47:43.644233 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:43.644238 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:43.645239 (+ 1001us) server_negotiation.cc:407] Connection header received
0806 18:47:43.646073 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:43.646076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:43.646123 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:43.646193 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:43.647570 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.648163 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.649325 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.649579 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.649680 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:43.652303 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:43.652306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:43.652308 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:43.652333 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:43.653633 (+ 1300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:43.654204 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:43.654206 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:43.654208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:43.654276 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:43.654630 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:43.654632 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:43.654633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:43.654794 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:43.654874 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:43.655468 (+ 594us) server_negotiation.cc:299] Negotiation successful
0806 18:47:43.655568 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:47:43.656894 24653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:43.644455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:60405)
0806 18:47:43.645094 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:43.645135 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:43.645887 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:43.646398 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:43.646416 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:43.646881 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:43.647427 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:43.647446 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.648286 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.648289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:43.649207 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:43.649218 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.649768 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.649772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:43.649946 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:43.650528 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:43.650545 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:43.652123 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:43.653750 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:43.653756 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:43.653772 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:43.654117 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:43.654364 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:43.654367 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:43.654369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:43.654545 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:43.654890 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:43.654897 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:43.655231 (+ 334us) client_negotiation.cc:769] Sending connection context
0806 18:47:43.655439 (+ 208us) client_negotiation.cc:240] Negotiation successful
0806 18:47:43.655780 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":119,"threads_started":1}
I20260806 18:47:43.658092 24649 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33973
I20260806 18:47:43.658358 24649 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:43.658906 24649 heartbeater.cc:511] Master 127.1.245.126:33973 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:47:43.660552 24360 ts_manager.cc:194] Registered new tserver with Master: 284aa6ffc0eb445a9ae22629f99238d8 (127.1.245.65:34507)
I20260806 18:47:43.661688 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:60405
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:47:43.674712 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33973
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:43.784448 24658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:43.784709 24658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:43.784768 24658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:43.788765 24658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:43.788829 24658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:43.788908 24658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:47:43.794108 24658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:43.795208 24658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:43.796267 24658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:43.805681 24665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:43.805719 24666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:43.805841 24670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:43.805878 24658 server_base.cc:1034] running on GCE node
I20260806 18:47:43.806464 24658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:43.807080 24658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:43.808249 24658 hybrid_clock.cc:648] HybridClock initialized: now 1786042063808229 us; error 31 us; skew 500 ppm
Aug 06 18:47:43 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042063, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:43.811087 24658 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:47:43.812014 24658 webserver.cc:460] Webserver started at http://127.1.245.66:41489/ using document root <none> and password file <none>
I20260806 18:47:43.812569 24658 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:43.812637 24658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:43.812836 24658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:43.814873 24658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "a07a75d32ef5488c98cd8a75a361c93f"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "3c4f2a11091121b156c989582bc5dd66"
server_key_iv: "5630aeb4d9b134f56c1d095eab519d6b"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.815408 24658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "a07a75d32ef5488c98cd8a75a361c93f"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "3c4f2a11091121b156c989582bc5dd66"
server_key_iv: "5630aeb4d9b134f56c1d095eab519d6b"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.819038 24658 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:47:43.821358 24683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:43.822608 24658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:47:43.822714 24658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "a07a75d32ef5488c98cd8a75a361c93f"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "3c4f2a11091121b156c989582bc5dd66"
server_key_iv: "5630aeb4d9b134f56c1d095eab519d6b"
server_key_version: "encryptionkey@0"
I20260806 18:47:43.822808 24658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:43.840075 24658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:43.843032 24658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:43.843227 24658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:43.843863 24658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:43.844736 24658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:43.844785 24658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:43.844841 24658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:43.844869 24658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:43.853683 24658 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:43889
I20260806 18:47:43.853714 24866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:43889 every 8 connection(s)
I20260806 18:47:43.854599 24658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:47:43.861181 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 24658
I20260806 18:47:43.861267 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:47:43.861514 2005 external_mini_cluster.cc:1428] Setting key 1665003b233b0b9b7ce3a37201eff74c
Aug 06 18:47:43 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042063, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:43.867518 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:43.856336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:55983 (local address 127.1.245.126:33973)
0806 18:47:43.856488 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:43.856493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:43.857593 (+ 1100us) server_negotiation.cc:407] Connection header received
0806 18:47:43.858525 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:43.858529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:43.858587 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:43.858655 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:43.859792 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.860311 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.861483 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.861729 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.861823 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:43.864198 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:43.864202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:43.864204 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:43.864231 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:43.865509 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:43.865986 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:43.865989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:43.865990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:43.866067 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:43.866391 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:43.866402 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:43.866404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:43.866582 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:43.866667 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:43.867225 (+ 558us) server_negotiation.cc:299] Negotiation successful
0806 18:47:43.867346 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 18:47:43.868351 24873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:43.856784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:55983)
0806 18:47:43.857396 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:43.857438 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:43.858283 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:43.858833 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:43.858839 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:43.859196 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:43.859651 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:43.859665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.860442 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.860445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:43.861366 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:43.861377 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:43.861868 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:43.861873 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:43.862046 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:43.862631 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:43.862651 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:43.864038 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:43.865620 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:43.865626 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:43.865641 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:43.865883 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:43.866147 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:43.866150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:43.866152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:43.866287 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:43.866684 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:43.866691 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:43.866964 (+ 273us) client_negotiation.cc:769] Sending connection context
0806 18:47:43.867148 (+ 184us) client_negotiation.cc:240] Negotiation successful
0806 18:47:43.867474 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":110,"threads_started":1}
I20260806 18:47:43.869575 24868 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33973
I20260806 18:47:43.869803 24868 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:43.870288 24868 heartbeater.cc:511] Master 127.1.245.126:33973 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:47:43.871381 24360 ts_manager.cc:194] Registered new tserver with Master: a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889)
I20260806 18:47:43.871977 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:55983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:47:43.889073 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33973
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:47:44.000576 24878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:47:44.000856 24878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:47:44.000918 24878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:47:44.005038 24878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:47:44.005105 24878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:47:44.005184 24878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:47:44.010454 24878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:47:44.011528 24878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:47:44.012553 24878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:47:44.022220 24884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:47:44.022265 24888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:44.022387 24878 server_base.cc:1034] running on GCE node
W20260806 18:47:44.022243 24885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:47:44.022938 24878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:47:44.023566 24878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:47:44.024734 24878 hybrid_clock.cc:648] HybridClock initialized: now 1786042064024719 us; error 36 us; skew 500 ppm
Aug 06 18:47:44 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042064, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:47:44.027601 24878 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:47:44.028523 24878 webserver.cc:460] Webserver started at http://127.1.245.67:44093/ using document root <none> and password file <none>
I20260806 18:47:44.029066 24878 fs_manager.cc:359] Metadata directory not provided
I20260806 18:47:44.029140 24878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:47:44.029382 24878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:47:44.031425 24878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "fe2fa64900e349db9829f7c73c93d528"
format_stamp: "Formatted at 2026-08-06 18:47:44 on dist-test-slave-fgb3"
server_key: "abef1e8162da6606d33e10ab615be191"
server_key_iv: "0ec469ba84aa7c3734ca90bd6173f28e"
server_key_version: "encryptionkey@0"
I20260806 18:47:44.031965 24878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "fe2fa64900e349db9829f7c73c93d528"
format_stamp: "Formatted at 2026-08-06 18:47:44 on dist-test-slave-fgb3"
server_key: "abef1e8162da6606d33e10ab615be191"
server_key_iv: "0ec469ba84aa7c3734ca90bd6173f28e"
server_key_version: "encryptionkey@0"
I20260806 18:47:44.035668 24878 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:47:44.038057 24898 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:44.039529 24878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:47:44.039662 24878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "fe2fa64900e349db9829f7c73c93d528"
format_stamp: "Formatted at 2026-08-06 18:47:44 on dist-test-slave-fgb3"
server_key: "abef1e8162da6606d33e10ab615be191"
server_key_iv: "0ec469ba84aa7c3734ca90bd6173f28e"
server_key_version: "encryptionkey@0"
I20260806 18:47:44.039776 24878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:47:44.071118 24878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:47:44.074499 24878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:47:44.074695 24878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:47:44.075354 24878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:47:44.076474 24878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:47:44.076540 24878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:44.076596 24878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:47:44.076625 24878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:47:44.085481 24878 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:36125
I20260806 18:47:44.085505 25069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:36125 every 8 connection(s)
I20260806 18:47:44.086493 24878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
Aug 06 18:47:44 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042064, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:47:44.095914 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 24878
I20260806 18:47:44.096032 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:47:44.096309 2005 external_mini_cluster.cc:1428] Setting key 81c534ab48f04c2cf9143a814b71cbbb
I20260806 18:47:44.100183 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.088328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:55751 (local address 127.1.245.126:33973)
0806 18:47:44.088455 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:44.088459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:44.089664 (+ 1205us) server_negotiation.cc:407] Connection header received
0806 18:47:44.090563 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:44.090566 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:44.090618 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:44.090694 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:44.091977 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.092502 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.093585 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.093786 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.093864 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.096510 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:44.096513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:44.096516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:44.096542 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:44.098026 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:44.098639 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:44.098643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:44.098645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:44.098723 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:44.099057 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:44.099060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:44.099061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:44.099245 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:44.099341 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:44.099870 (+ 529us) server_negotiation.cc:299] Negotiation successful
0806 18:47:44.100009 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:47:44.100978 25072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.088690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:55751)
0806 18:47:44.089491 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:44.089537 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:44.090365 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:44.090921 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:44.090928 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:44.091311 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:47:44.091847 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.091861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.092598 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.092600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.093492 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.093501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.093928 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.093931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.094090 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.094712 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:47:44.094729 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:47:44.096325 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:47:44.098136 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:44.098142 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:44.098157 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:44.098545 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:44.098808 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:47:44.098811 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:47:44.098813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:47:44.098959 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:47:44.099357 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:47:44.099364 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:47:44.099620 (+ 256us) client_negotiation.cc:769] Sending connection context
0806 18:47:44.099792 (+ 172us) client_negotiation.cc:240] Negotiation successful
0806 18:47:44.100127 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":119,"threads_started":1}
I20260806 18:47:44.102025 25070 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33973
I20260806 18:47:44.102245 25070 heartbeater.cc:464] Registering TS with master...
I20260806 18:47:44.102782 25070 heartbeater.cc:511] Master 127.1.245.126:33973 requested a full tablet report, sending...
I20260806 18:47:44.103797 24360 ts_manager.cc:194] Registered new tserver with Master: fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125)
I20260806 18:47:44.104388 24360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:55751
I20260806 18:47:44.110121 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:47:44.118783 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.112020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.1.245.126:33973)
0806 18:47:44.112146 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:44.112151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:44.112333 (+ 182us) server_negotiation.cc:407] Connection header received
0806 18:47:44.112452 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:44.112456 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:44.112502 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:44.112566 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:47:44.113227 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.113729 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.114605 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.114819 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.114905 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.116133 (+ 1228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:47:44.116136 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:47:44.116138 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:47:44.116161 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:47:44.117192 (+ 1031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:44.117579 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:44.117583 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:44.117584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:44.117660 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:47:44.117901 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:47:44.117904 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:47:44.117906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:47:44.118138 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:47:44.118260 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:44.118527 (+ 267us) server_negotiation.cc:299] Negotiation successful
0806 18:47:44.118635 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:47:44.121280 24360 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35540:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:47:44.124334 24360 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:47:44.138263 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.132989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58120 (local address 127.1.245.67:36125)
0806 18:47:44.133531 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:44.133541 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:44.133559 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:47:44.133632 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:44.133637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:44.133834 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:44.133990 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:44.134268 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.135089 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.136827 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.137613 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.137725 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.137935 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:44.138058 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 18:47:44.138138 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":139,"threads_started":1}
I20260806 18:47:44.138307 25092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.132489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:36125 (local address 127.0.0.1:58120)
0806 18:47:44.133013 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:44.133068 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:44.133204 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:44.133988 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:44.133994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:44.134017 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:44.134156 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.134163 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.135214 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.135218 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.136692 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.136702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.137732 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.137735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.137843 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.137891 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 18:47:44.138034 (+ 143us) client_negotiation.cc:240] Negotiation successful
0806 18:47:44.138153 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":51,"spinlock_wait_cycles":2560,"thread_start_us":90,"threads_started":1}
I20260806 18:47:44.139169 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.132527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:43889 (local address 127.0.0.1:49102)
0806 18:47:44.133412 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:44.133423 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:44.133522 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:44.134175 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:44.134178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:44.134206 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:44.135292 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.135299 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.136697 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.136700 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.137435 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.137444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.138685 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.138688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.138820 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.138861 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 18:47:44.138932 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:47:44.139003 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":830,"mutex_wait_us":1015,"spinlock_wait_cycles":2176,"thread_start_us":91,"threads_started":1}
I20260806 18:47:44.139168 25086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.132224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34507 (local address 127.0.0.1:52794)
0806 18:47:44.133013 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:44.133068 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:44.133204 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:44.134177 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:44.134180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:44.134206 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:44.134349 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.134356 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.135355 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.135359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.136666 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.136676 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.138674 (+ 1998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.138678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.138817 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.138855 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 18:47:44.138932 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:47:44.139003 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"mutex_wait_us":97,"spinlock_wait_cycles":12288,"thread_start_us":67,"threads_started":1}
I20260806 18:47:44.139883 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.132989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49102 (local address 127.1.245.66:43889)
0806 18:47:44.133512 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:44.133525 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:44.133552 (+ 27us) server_negotiation.cc:407] Connection header received
0806 18:47:44.133632 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:44.133637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:44.133834 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:44.133991 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:44.135424 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.136551 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.137562 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.138389 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.138493 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.138634 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:44.139642 (+ 1008us) server_negotiation.cc:299] Negotiation successful
0806 18:47:44.139744 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":122,"threads_started":1}
I20260806 18:47:44.140161 25087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.132419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52794 (local address 127.1.245.65:34507)
0806 18:47:44.133748 (+ 1329us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:44.133765 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:44.133789 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:47:44.133866 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:44.133871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:44.134061 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:44.134209 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:44.134473 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.135213 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.136779 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.137613 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.137725 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.137887 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:44.138984 (+ 1097us) server_negotiation.cc:299] Negotiation successful
0806 18:47:44.139081 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1222,"thread_start_us":121,"threads_started":1}
I20260806 18:47:44.141480 24974 tablet_service.cc:1467] Processing CreateTablet for tablet 8d123cfd92454825ba0921847edc2e87 (DEFAULT_TABLE table=test-table [id=accc5c2500da4e1da14625c2a2b7c02d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:44.142717 24974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d123cfd92454825ba0921847edc2e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:44.142518 24766 tablet_service.cc:1467] Processing CreateTablet for tablet 8d123cfd92454825ba0921847edc2e87 (DEFAULT_TABLE table=test-table [id=accc5c2500da4e1da14625c2a2b7c02d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:44.143549 24766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d123cfd92454825ba0921847edc2e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:44.145674 24548 tablet_service.cc:1467] Processing CreateTablet for tablet 8d123cfd92454825ba0921847edc2e87 (DEFAULT_TABLE table=test-table [id=accc5c2500da4e1da14625c2a2b7c02d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:47:44.146579 24548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8d123cfd92454825ba0921847edc2e87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:47:44.151029 25097 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Bootstrap starting.
I20260806 18:47:44.151481 25098 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Bootstrap starting.
I20260806 18:47:44.153137 25101 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Bootstrap starting.
I20260806 18:47:44.153849 25098 tablet_bootstrap.cc:654] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:44.155004 25098 log.cc:824] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:44.155372 25101 tablet_bootstrap.cc:654] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:44.156579 25101 log.cc:824] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:44.157397 25098 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: No bootstrap required, opened a new log
I20260806 18:47:44.157617 25097 tablet_bootstrap.cc:654] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Neither blocks nor log segments found. Creating new log.
I20260806 18:47:44.157616 25098 ts_tablet_manager.cc:1397] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:47:44.158283 25097 log.cc:824] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:47:44.158733 25101 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: No bootstrap required, opened a new log
I20260806 18:47:44.158919 25101 ts_tablet_manager.cc:1397] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:47:44.159904 25097 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: No bootstrap required, opened a new log
I20260806 18:47:44.160073 25097 ts_tablet_manager.cc:1397] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260806 18:47:44.161592 25098 raft_consensus.cc:348] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.161890 25098 raft_consensus.cc:374] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:44.161958 25098 raft_consensus.cc:729] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2fa64900e349db9829f7c73c93d528, State: Initialized, Role: FOLLOWER
I20260806 18:47:44.162482 25098 consensus_queue.cc:260] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.162642 25101 raft_consensus.cc:348] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.162891 25101 raft_consensus.cc:374] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:44.162959 25101 raft_consensus.cc:729] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284aa6ffc0eb445a9ae22629f99238d8, State: Initialized, Role: FOLLOWER
I20260806 18:47:44.163323 25070 heartbeater.cc:503] Master 127.1.245.126:33973 was elected leader, sending a full tablet report...
I20260806 18:47:44.163400 25101 consensus_queue.cc:260] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.163376 25097 raft_consensus.cc:348] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.163578 25098 ts_tablet_manager.cc:1428] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:47:44.163621 25097 raft_consensus.cc:374] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:47:44.163667 25097 raft_consensus.cc:729] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07a75d32ef5488c98cd8a75a361c93f, State: Initialized, Role: FOLLOWER
I20260806 18:47:44.164054 25097 consensus_queue.cc:260] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.164227 25101 ts_tablet_manager.cc:1428] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:47:44.164565 24649 heartbeater.cc:503] Master 127.1.245.126:33973 was elected leader, sending a full tablet report...
I20260806 18:47:44.164963 25097 ts_tablet_manager.cc:1428] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:47:44.165299 24868 heartbeater.cc:503] Master 127.1.245.126:33973 was elected leader, sending a full tablet report...
W20260806 18:47:44.337546 25071 tablet.cc:2369] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:44.348489 25110 raft_consensus.cc:482] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:47:44.348732 25110 raft_consensus.cc:504] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.349905 25110 leader_election.cc:288] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125), a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889)
I20260806 18:47:44.355216 25094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.350300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:46967 (local address 127.1.245.67:36125)
0806 18:47:44.350464 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:44.350469 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:44.350712 (+ 243us) server_negotiation.cc:407] Connection header received
0806 18:47:44.350898 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:44.350901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:44.350959 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:44.351039 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:44.351463 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.352218 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.353957 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.354590 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.354709 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.354762 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:44.355014 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 18:47:44.355083 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260806 18:47:44.355216 25093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.350297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:49739 (local address 127.1.245.66:43889)
0806 18:47:44.350459 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:44.350464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:44.350769 (+ 305us) server_negotiation.cc:407] Connection header received
0806 18:47:44.350938 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:44.350942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:44.350993 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:44.351064 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:47:44.351435 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.352224 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.353957 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.354589 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.354709 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.354762 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:44.355014 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 18:47:44.355087 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 18:47:44.355274 25113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.350173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:36125 (local address 127.1.245.65:46967)
0806 18:47:44.350617 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:44.350632 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:44.350772 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:44.351062 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:44.351065 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:44.351109 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:44.351316 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.351324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.352326 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.352329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.353836 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.353848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.354714 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.354718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.354854 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.354911 (+ 57us) client_negotiation.cc:769] Sending connection context
0806 18:47:44.354979 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:47:44.355079 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":247,"spinlock_wait_cycles":39168,"thread_start_us":158,"threads_started":1}
I20260806 18:47:44.355455 25112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.350188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:43889 (local address 127.1.245.65:49739)
0806 18:47:44.350678 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:44.350689 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:47:44.350818 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:47:44.351071 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:47:44.351073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:47:44.351105 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:47:44.351326 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.351332 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.352326 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.352329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.353839 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:47:44.353848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.354714 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.354718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:47:44.354850 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.354905 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 18:47:44.354979 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:47:44.355079 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":205,"spinlock_wait_cycles":21120,"thread_start_us":132,"threads_started":1}
W20260806 18:47:44.355736 24872 tablet.cc:2369] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:44.355995 25005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d123cfd92454825ba0921847edc2e87" candidate_uuid: "284aa6ffc0eb445a9ae22629f99238d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2fa64900e349db9829f7c73c93d528" is_pre_election: true
I20260806 18:47:44.356192 24793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d123cfd92454825ba0921847edc2e87" candidate_uuid: "284aa6ffc0eb445a9ae22629f99238d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07a75d32ef5488c98cd8a75a361c93f" is_pre_election: true
I20260806 18:47:44.356326 25005 raft_consensus.cc:2454] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 284aa6ffc0eb445a9ae22629f99238d8 in term 0.
I20260806 18:47:44.356470 24793 raft_consensus.cc:2454] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 284aa6ffc0eb445a9ae22629f99238d8 in term 0.
I20260806 18:47:44.356884 24476 leader_election.cc:302] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284aa6ffc0eb445a9ae22629f99238d8, fe2fa64900e349db9829f7c73c93d528; no voters:
I20260806 18:47:44.357182 25110 raft_consensus.cc:2781] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:47:44.357282 25110 raft_consensus.cc:482] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:47:44.357314 25110 raft_consensus.cc:3037] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:44.358491 25110 raft_consensus.cc:504] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.358841 25110 leader_election.cc:288] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 1 election: Requested vote from peers fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125), a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889)
I20260806 18:47:44.359218 24793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d123cfd92454825ba0921847edc2e87" candidate_uuid: "284aa6ffc0eb445a9ae22629f99238d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a07a75d32ef5488c98cd8a75a361c93f"
I20260806 18:47:44.359220 25005 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8d123cfd92454825ba0921847edc2e87" candidate_uuid: "284aa6ffc0eb445a9ae22629f99238d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2fa64900e349db9829f7c73c93d528"
I20260806 18:47:44.359337 25005 raft_consensus.cc:3037] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:44.359331 24793 raft_consensus.cc:3037] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:47:44.360355 24793 raft_consensus.cc:2454] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 284aa6ffc0eb445a9ae22629f99238d8 in term 1.
I20260806 18:47:44.360363 25005 raft_consensus.cc:2454] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 284aa6ffc0eb445a9ae22629f99238d8 in term 1.
I20260806 18:47:44.360692 24478 leader_election.cc:302] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284aa6ffc0eb445a9ae22629f99238d8, a07a75d32ef5488c98cd8a75a361c93f; no voters:
I20260806 18:47:44.360870 25110 raft_consensus.cc:2781] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:47:44.361102 25110 raft_consensus.cc:686] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 LEADER]: Becoming Leader. State: Replica: 284aa6ffc0eb445a9ae22629f99238d8, State: Running, Role: LEADER
I20260806 18:47:44.361482 25110 consensus_queue.cc:237] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:47:44.364769 24359 catalog_manager.cc:5495] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 284aa6ffc0eb445a9ae22629f99238d8 (127.1.245.65). New cstate: current_term: 1 leader_uuid: "284aa6ffc0eb445a9ae22629f99238d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } health_report { overall_health: UNKNOWN } } }
W20260806 18:47:44.393711 24652 tablet.cc:2369] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:47:44.429322 25087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.426076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52796 (local address 127.1.245.65:34507)
0806 18:47:44.426207 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:47:44.426211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:47:44.426224 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:47:44.426303 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:47:44.426306 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:47:44.426405 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:47:44.426492 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:47:44.426745 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.427264 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.428066 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:47:44.428269 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:47:44.428342 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:47:44.428572 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:47:44.428983 (+ 411us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:47:44.429074 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:47:44.429131 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 18:47:44.429181 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:47:44.435112 25005 raft_consensus.cc:1261] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 1 FOLLOWER]: Refusing update from remote peer 284aa6ffc0eb445a9ae22629f99238d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:44.435112 24793 raft_consensus.cc:1261] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 1 FOLLOWER]: Refusing update from remote peer 284aa6ffc0eb445a9ae22629f99238d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:47:44.435778 25114 consensus_queue.cc:1035] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:44.435972 25110 consensus_queue.cc:1035] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:47:44.445263 25123 mvcc.cc:205] Tried to move back new op lower bound from 7315628295918854144 to 7315628295627644928. Current Snapshot: MvccSnapshot[applied={T|T < 7315628295918854144 or (T in {7315628295918854144})}]
I20260806 18:47:44.453166 24359 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:47:44.453315 24359 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35540:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:47:44.455634 24359 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Sending DeleteTablet for 3 replicas of tablet 8d123cfd92454825ba0921847edc2e87
I20260806 18:47:44.456357 24548 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d123cfd92454825ba0921847edc2e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:44 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:52794
I20260806 18:47:44.456401 24974 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d123cfd92454825ba0921847edc2e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:44 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:58120
I20260806 18:47:44.456539 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 24455
I20260806 18:47:44.456501 24766 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d123cfd92454825ba0921847edc2e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:44 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:49102
I20260806 18:47:44.463670 25142 tablet_replica.cc:330] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: stopping tablet replica
I20260806 18:47:44.464007 25142 raft_consensus.cc:2229] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:47:44.464298 25142 raft_consensus.cc:2258] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:44.466130 25143 tablet_replica.cc:330] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: stopping tablet replica
I20260806 18:47:44.466270 25142 ts_tablet_manager.cc:1905] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:44.466482 25143 raft_consensus.cc:2229] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:47:44.466802 25143 raft_consensus.cc:2258] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:47:44.468477 25143 ts_tablet_manager.cc:1905] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:47:44.469403 25142 ts_tablet_manager.cc:1918] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:44.469522 25142 log.cc:1196] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/8d123cfd92454825ba0921847edc2e87
I20260806 18:47:44.469903 25142 ts_tablet_manager.cc:1939] T 8d123cfd92454825ba0921847edc2e87 P fe2fa64900e349db9829f7c73c93d528: Deleting consensus metadata
I20260806 18:47:44.471302 25143 ts_tablet_manager.cc:1918] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:47:44.471398 25143 log.cc:1196] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/8d123cfd92454825ba0921847edc2e87
I20260806 18:47:44.471374 24334 catalog_manager.cc:4841] TS fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125): tablet 8d123cfd92454825ba0921847edc2e87 (table test-table [id=accc5c2500da4e1da14625c2a2b7c02d]) successfully deleted
W20260806 18:47:44.471526 24336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:34507 (error 108)
I20260806 18:47:44.471729 25143 ts_tablet_manager.cc:1939] T 8d123cfd92454825ba0921847edc2e87 P a07a75d32ef5488c98cd8a75a361c93f: Deleting consensus metadata
I20260806 18:47:44.472316 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 24658
I20260806 18:47:44.472579 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.472277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34507 (local address 127.0.0.1:52806)
0806 18:47:44.472425 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:44.472495 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 18:47:44.474445 24336 catalog_manager.cc:4584] TS 284aa6ffc0eb445a9ae22629f99238d8 (127.1.245.65:34507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8d123cfd92454825ba0921847edc2e87: Network error: Client connection negotiation failed: client connection to 127.1.245.65:34507: connect: Connection refused (error 111)
I20260806 18:47:44.481892 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 24878
I20260806 18:47:44.485174 25089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:47:44.482097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:43889 (local address 127.0.0.1:49118)
0806 18:47:44.485005 (+ 2908us) negotiation.cc:107] Waiting for socket to connect
0806 18:47:44.485081 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.66:43889: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2818}
I20260806 18:47:44.493938 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 24310
I20260806 18:48:04.505044 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33973
--webserver_interface=127.1.245.126
--webserver_port=43973
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:33973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:04.617934 25145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:04.618196 25145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:04.618234 25145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:04.622124 25145 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:48:04.622172 25145 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:48:04.622187 25145 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:48:04.622200 25145 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:04.622215 25145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:04.622232 25145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:04.622252 25145 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:04.622287 25145 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:04.627458 25145 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:33973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=43973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:04.628456 25145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:04.629434 25145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:48:04.643101 25145 server_base.cc:1034] running on GCE node
W20260806 18:48:04.643123 25163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:04.643127 25160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:04.643141 25159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:04.644104 25145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:04.645211 25145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:04.646394 25145 hybrid_clock.cc:648] HybridClock initialized: now 1786042084646367 us; error 33 us; skew 500 ppm
Aug 06 18:48:04 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042084, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:04.653473 25145 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:48:04.654508 25145 webserver.cc:460] Webserver started at http://127.1.245.126:43973/ using document root <none> and password file <none>
I20260806 18:48:04.655088 25145 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:04.655171 25145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:04.659261 25145 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 18:48:04.661521 25173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:04.662890 25145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:04.663031 25145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "3ac19ea0977f4861bf1e52218e8943a7"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "b77c7ccc2c090aa7e6de6183d86adc6a"
server_key_iv: "360c6fc625ece6a29c60d426b44119b2"
server_key_version: "encryptionkey@0"
I20260806 18:48:04.663661 25145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:04.675020 25145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:04.678084 25145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:04.678282 25145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:04.685413 25145 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:33973
I20260806 18:48:04.685701 25258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:33973 every 8 connection(s)
I20260806 18:48:04.686530 25145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:04.691318 25259 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:48:04.692610 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 25145
I20260806 18:48:04.693030 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:34507
--local_ip_for_outbound_sockets=127.1.245.65
--tserver_master_addrs=127.1.245.126:33973
--webserver_port=41373
--webserver_interface=127.1.245.65
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:48:04.693164 25259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Bootstrap starting.
I20260806 18:48:04.710613 25259 log.cc:824] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:04.719884 25259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:48:04.720258 25259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Bootstrap complete.
I20260806 18:48:04.724318 25259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:48:04.724704 25259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac19ea0977f4861bf1e52218e8943a7, State: Initialized, Role: FOLLOWER
I20260806 18:48:04.725160 25259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:48:04.725296 25259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:04.725368 25259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:04.725457 25259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 1 FOLLOWER]: Advancing to term 2
I20260806 18:48:04.727388 25259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:48:04.727722 25259 leader_election.cc:302] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ac19ea0977f4861bf1e52218e8943a7; no voters:
I20260806 18:48:04.728016 25259 leader_election.cc:288] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 18:48:04.728143 25267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 18:48:04.728399 25267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 2 LEADER]: Becoming Leader. State: Replica: 3ac19ea0977f4861bf1e52218e8943a7, State: Running, Role: LEADER
I20260806 18:48:04.728703 25267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:48:04.729185 25259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:04.730099 25269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ac19ea0977f4861bf1e52218e8943a7. Latest consensus state: current_term: 2 leader_uuid: "3ac19ea0977f4861bf1e52218e8943a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } } }
I20260806 18:48:04.730232 25269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:04.730513 25268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3ac19ea0977f4861bf1e52218e8943a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } } }
I20260806 18:48:04.730599 25268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:04.734208 25290 catalog_manager.cc:1259] Loaded cluster ID: ec4094cab337420896a619c4b681e44c
I20260806 18:48:04.734268 25290 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: loading cluster ID for follower catalog manager: success
I20260806 18:48:04.736756 25290 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: acquiring CA information for follower catalog manager: success
I20260806 18:48:04.738338 25290 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:48:04.738479 25280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:04.740398 25280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:04.740790 25280 catalog_manager.cc:1259] Loaded cluster ID: ec4094cab337420896a619c4b681e44c
I20260806 18:48:04.740839 25280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:04.741374 25280 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:04.741778 25280 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Loaded TSK: 0
I20260806 18:48:04.751169 25280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Generated new TSK 1
I20260806 18:48:04.751448 25280 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:48:04.822050 25263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:04.822386 25263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:04.822459 25263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:04.826582 25263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:04.826681 25263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:04.826767 25263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:48:04.832520 25263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:34507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=41373
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:04.833583 25263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:04.834681 25263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:04.844726 25300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:04.844779 25302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:04.844710 25305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:04.844938 25263 server_base.cc:1034] running on GCE node
I20260806 18:48:04.845580 25263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:04.846304 25263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:04.847522 25263 hybrid_clock.cc:648] HybridClock initialized: now 1786042084847507 us; error 28 us; skew 500 ppm
Aug 06 18:48:04 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042084, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:04.850425 25263 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:48:04.851423 25263 webserver.cc:460] Webserver started at http://127.1.245.65:41373/ using document root <none> and password file <none>
I20260806 18:48:04.851996 25263 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:04.852083 25263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:04.856261 25263 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260806 18:48:04.858600 25312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:04.859869 25263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:48:04.860080 25263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "284aa6ffc0eb445a9ae22629f99238d8"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "e5b9964aa3e029703c981b9f275aa96a"
server_key_iv: "26e8e9edd21e9f59eb2b2d9c0c0094f9"
server_key_version: "encryptionkey@0"
I20260806 18:48:04.860630 25263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:04.875188 25263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:04.878238 25263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:04.878450 25263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:04.879086 25263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:04.880225 25319 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:48:04.884363 25263 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:48:04.884420 25263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260806 18:48:04.884471 25263 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:48:04.886116 25263 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:48:04.886173 25263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260806 18:48:04.886480 25319 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Bootstrap starting.
I20260806 18:48:04.896842 25263 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:34507
I20260806 18:48:04.896850 25482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:34507 every 8 connection(s)
I20260806 18:48:04.898160 25263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:48:04.900857 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 25263
I20260806 18:48:04.901346 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:43889
--local_ip_for_outbound_sockets=127.1.245.66
--tserver_master_addrs=127.1.245.126:33973
--webserver_port=41489
--webserver_interface=127.1.245.66
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:48:04.908005 25319 log.cc:824] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:48:04 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042084, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:04.916564 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:04.900456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:35269)
0806 18:48:04.901190 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:04.901251 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:04.902400 (+ 1149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:04.903630 (+ 1230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:04.903638 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:04.904051 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:04.904511 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:04.904525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:04.905837 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:04.905840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:04.906772 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:04.906782 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:04.907729 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:04.907733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:04.907939 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:04.908609 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:04.908626 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:04.910409 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:04.912914 (+ 2505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:04.912921 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:04.912935 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:04.913293 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:04.913723 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:04.913726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:04.913728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:04.913921 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:04.914676 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:04.914682 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:04.915006 (+ 324us) client_negotiation.cc:769] Sending connection context
0806 18:48:04.915221 (+ 215us) client_negotiation.cc:240] Negotiation successful
0806 18:48:04.915595 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":171,"threads_started":1}
I20260806 18:48:04.916913 25319 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:48:04.917310 25319 tablet_bootstrap.cc:492] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Bootstrap complete.
I20260806 18:48:04.917569 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:04.900506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:35269 (local address 127.1.245.126:33973)
0806 18:48:04.901853 (+ 1347us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:04.901873 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:04.901922 (+ 49us) server_negotiation.cc:407] Connection header received
0806 18:48:04.902922 (+ 1000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:04.902933 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:04.903245 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:04.903552 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:04.904680 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:04.905675 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:04.906888 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:04.907559 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:04.907736 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:04.910642 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:04.910649 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:04.910666 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:04.910701 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:04.912779 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:04.913445 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:04.913451 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:04.913456 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:04.913608 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:04.914031 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:04.914034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:04.914036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:04.914545 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:04.914690 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:04.915386 (+ 696us) server_negotiation.cc:299] Negotiation successful
0806 18:48:04.915686 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1181,"thread_start_us":128,"threads_started":1}
I20260806 18:48:04.918705 25319 ts_tablet_manager.cc:1397] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.018s
I20260806 18:48:04.919909 25483 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33973
I20260806 18:48:04.920148 25483 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:04.920714 25483 heartbeater.cc:511] Master 127.1.245.126:33973 requested a full tablet report, sending...
I20260806 18:48:04.921392 25319 raft_consensus.cc:348] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:48:04.922029 25319 raft_consensus.cc:729] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284aa6ffc0eb445a9ae22629f99238d8, State: Initialized, Role: FOLLOWER
I20260806 18:48:04.922539 25319 consensus_queue.cc:260] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } } peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } }
I20260806 18:48:04.923566 25319 ts_tablet_manager.cc:1428] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:48:04.923777 25204 ts_manager.cc:194] Registered new tserver with Master: 284aa6ffc0eb445a9ae22629f99238d8 (127.1.245.65:34507)
I20260806 18:48:04.927749 25204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:35269
I20260806 18:48:04.930549 25483 heartbeater.cc:503] Master 127.1.245.126:33973 was elected leader, sending a full tablet report...
Aug 06 18:48:04 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042084, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.65@KRBTEST.COM
I20260806 18:48:04.938961 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:04.927055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34507 (local address 127.0.0.1:46376)
0806 18:48:04.927455 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:04.927499 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:04.927626 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:04.928603 (+ 977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:04.928608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:04.929009 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:04.929157 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:04.929166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:04.930873 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:04.930876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:04.931893 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:04.931906 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:04.932669 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:04.932672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:04.932758 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:04.933336 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:04.933348 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:04.934952 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:04.937062 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:04.937073 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:04.937078 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:04.937370 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:04.937682 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:04.937685 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:04.937686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:04.937784 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:04.938362 (+ 578us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:04.938369 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:04.938488 (+ 119us) client_negotiation.cc:769] Sending connection context
0806 18:48:04.938619 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 18:48:04.938801 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":108,"threads_started":1}
I20260806 18:48:04.939036 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:04.927811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46376 (local address 127.1.245.65:34507)
0806 18:48:04.928142 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:04.928154 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:04.928178 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:48:04.928245 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:04.928250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:04.928434 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:04.929379 (+ 945us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:04.929652 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:04.930739 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:04.932051 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:04.932550 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:04.932656 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:04.935111 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:04.935122 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:04.935135 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:04.935163 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:04.936933 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:04.937487 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:04.937492 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:04.937494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:04.937589 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:04.937867 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:04.937870 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:04.937872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:04.938171 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:04.938345 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:04.938711 (+ 366us) server_negotiation.cc:299] Negotiation successful
0806 18:48:04.938869 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":112,"threads_started":1}
I20260806 18:48:04.940090 25386 tablet_service.cc:1514] Processing DeleteTablet for tablet 8d123cfd92454825ba0921847edc2e87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:47:44 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:46376
I20260806 18:48:04.941118 25499 tablet_replica.cc:330] stopping tablet replica
I20260806 18:48:04.941416 25499 raft_consensus.cc:2229] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:04.941654 25499 raft_consensus.cc:2258] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:04.942652 25499 ts_tablet_manager.cc:1905] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:04.946138 25499 ts_tablet_manager.cc:1918] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:04.946231 25499 log.cc:1196] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal/wals/8d123cfd92454825ba0921847edc2e87
I20260806 18:48:04.946502 25499 ts_tablet_manager.cc:1939] T 8d123cfd92454825ba0921847edc2e87 P 284aa6ffc0eb445a9ae22629f99238d8: Deleting consensus metadata
I20260806 18:48:04.948208 25179 catalog_manager.cc:4841] TS 284aa6ffc0eb445a9ae22629f99238d8 (127.1.245.65:34507): tablet 8d123cfd92454825ba0921847edc2e87 (table test-table [id=accc5c2500da4e1da14625c2a2b7c02d]) successfully deleted
W20260806 18:48:05.027383 25492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:05.027688 25492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:05.027734 25492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:05.032111 25492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:05.032191 25492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:05.032289 25492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:48:05.038686 25492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:43889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=41489
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:05.039885 25492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:05.040966 25492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:05.051112 25507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:05.051168 25511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:05.051131 25509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:05.051568 25492 server_base.cc:1034] running on GCE node
I20260806 18:48:05.052026 25492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:05.052717 25492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:05.053917 25492 hybrid_clock.cc:648] HybridClock initialized: now 1786042085053891 us; error 36 us; skew 500 ppm
Aug 06 18:48:05 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042085, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:05.056844 25492 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:48:05.057798 25492 webserver.cc:460] Webserver started at http://127.1.245.66:41489/ using document root <none> and password file <none>
I20260806 18:48:05.058459 25492 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:05.058555 25492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:05.062639 25492 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:48:05.065023 25521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:05.066421 25492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:48:05.066552 25492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "a07a75d32ef5488c98cd8a75a361c93f"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "3c4f2a11091121b156c989582bc5dd66"
server_key_iv: "5630aeb4d9b134f56c1d095eab519d6b"
server_key_version: "encryptionkey@0"
I20260806 18:48:05.067178 25492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:05.093737 25492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:05.096859 25492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:05.097051 25492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:05.097680 25492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:05.098624 25492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:05.098676 25492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:05.098731 25492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:05.098762 25492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:05.107218 25492 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:43889
I20260806 18:48:05.107267 25689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:43889 every 8 connection(s)
I20260806 18:48:05.108242 25492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:48:05.111496 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 25492
I20260806 18:48:05.111979 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:36125
--local_ip_for_outbound_sockets=127.1.245.67
--tserver_master_addrs=127.1.245.126:33973
--webserver_port=44093
--webserver_interface=127.1.245.67
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:48:05 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042085, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:05.121785 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.110166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:48813 (local address 127.1.245.126:33973)
0806 18:48:05.110351 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.110356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.111491 (+ 1135us) server_negotiation.cc:407] Connection header received
0806 18:48:05.112510 (+ 1019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.112513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.112579 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.112654 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:05.113847 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.114430 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.115578 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.115820 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.115916 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.118441 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:05.118445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:05.118447 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:05.118473 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:05.119806 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:05.120289 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:05.120292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:05.120293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:05.120381 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:05.120693 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:05.120697 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:05.120699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:05.120880 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:05.120968 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.121469 (+ 501us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.121613 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:48:05.122651 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.110623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:48813)
0806 18:48:05.111342 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:05.111386 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:05.112290 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:05.112850 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:05.112857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:05.113211 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:05.113690 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.113702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.114558 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.114560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.115459 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.115469 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.115964 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.115968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.116137 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.116808 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:05.116825 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:05.118249 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:05.119917 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:05.119924 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:05.119939 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:05.120193 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:05.120465 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:05.120468 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:05.120470 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:05.120608 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:05.120980 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:05.120986 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:05.121234 (+ 248us) client_negotiation.cc:769] Sending connection context
0806 18:48:05.121414 (+ 180us) client_negotiation.cc:240] Negotiation successful
0806 18:48:05.121744 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":125,"threads_started":1}
I20260806 18:48:05.123800 25690 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33973
I20260806 18:48:05.124034 25690 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:05.124524 25690 heartbeater.cc:511] Master 127.1.245.126:33973 requested a full tablet report, sending...
I20260806 18:48:05.125526 25204 ts_manager.cc:194] Registered new tserver with Master: a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889)
I20260806 18:48:05.126359 25204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:48813
W20260806 18:48:05.225358 25693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:05.225620 25693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:05.225663 25693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:05.229727 25693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:05.229797 25693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:05.229867 25693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:48:05.235144 25693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:36125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=44093
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:05.236179 25693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:05.237175 25693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:05.247120 25700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:05.247123 25702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:05.247450 25693 server_base.cc:1034] running on GCE node
W20260806 18:48:05.247126 25698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:05.248055 25693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:05.248700 25693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:05.249874 25693 hybrid_clock.cc:648] HybridClock initialized: now 1786042085249855 us; error 29 us; skew 500 ppm
Aug 06 18:48:05 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042085, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:05.252713 25693 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:48:05.253628 25693 webserver.cc:460] Webserver started at http://127.1.245.67:44093/ using document root <none> and password file <none>
I20260806 18:48:05.254199 25693 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:05.254298 25693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:05.258342 25693 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260806 18:48:05.260399 25716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:05.261602 25693 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:48:05.261726 25693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "fe2fa64900e349db9829f7c73c93d528"
format_stamp: "Formatted at 2026-08-06 18:47:44 on dist-test-slave-fgb3"
server_key: "abef1e8162da6606d33e10ab615be191"
server_key_iv: "0ec469ba84aa7c3734ca90bd6173f28e"
server_key_version: "encryptionkey@0"
I20260806 18:48:05.262264 25693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:05.283156 25693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:05.286450 25693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:05.286644 25693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:05.287277 25693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:05.288147 25693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:05.288197 25693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:05.288254 25693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:05.288283 25693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:05.297246 25693 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:36125
I20260806 18:48:05.297272 25883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:36125 every 8 connection(s)
I20260806 18:48:05.298345 25693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:48:05.298980 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 25693
I20260806 18:48:05.308173 25487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.299811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54506 (local address 127.1.245.126:33973)
0806 18:48:05.299970 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.299975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.300162 (+ 187us) server_negotiation.cc:407] Connection header received
0806 18:48:05.300346 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.300350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.300404 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.300484 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:05.301396 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.301921 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.302906 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.303186 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.303286 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.305008 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:05.305012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:05.305014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:05.305044 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:05.306546 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:05.307006 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:05.307009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:05.307010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:05.307087 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:05.307330 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:05.307333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:05.307335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:05.307551 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:05.307731 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.307911 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.308017 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 06 18:48:05 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042085, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:05.314081 25889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.300336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:43561 (local address 127.1.245.126:33973)
0806 18:48:05.300615 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.300618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.302123 (+ 1505us) server_negotiation.cc:407] Connection header received
0806 18:48:05.303073 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.303076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.303139 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.303245 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:05.304891 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.305673 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.307164 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.307461 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.307564 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.310443 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:05.310446 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:05.310449 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:05.310471 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:05.311722 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:05.312366 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:05.312369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:05.312370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:05.312454 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:05.312848 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:05.312851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:05.312852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:05.313044 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:05.313119 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.313770 (+ 651us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.313903 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":99,"threads_started":1}
I20260806 18:48:05.315235 25890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.300936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:43561)
0806 18:48:05.301967 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:05.302020 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:05.302889 (+ 869us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:05.303499 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:05.303509 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:05.303983 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:05.304705 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.304726 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.305833 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.305837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.307054 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.307064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.307625 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.307630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.307840 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.308474 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:05.308490 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:05.310256 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:05.311842 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:05.311849 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:05.311883 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:05.312246 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:05.312553 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:05.312557 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:05.312559 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:05.312753 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:05.313146 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:05.313153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:05.313498 (+ 345us) client_negotiation.cc:769] Sending connection context
0806 18:48:05.313711 (+ 213us) client_negotiation.cc:240] Negotiation successful
0806 18:48:05.314092 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"thread_start_us":625,"threads_started":1}
I20260806 18:48:05.317413 25884 heartbeater.cc:347] Connected to a master server at 127.1.245.126:33973
I20260806 18:48:05.317693 25884 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:05.318241 25884 heartbeater.cc:511] Master 127.1.245.126:33973 requested a full tablet report, sending...
I20260806 18:48:05.319371 25204 ts_manager.cc:194] Registered new tserver with Master: fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125)
I20260806 18:48:05.319934 25204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:43561
I20260806 18:48:05.322103 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:48:05.325815 25889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.322741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.1.245.126:33973)
0806 18:48:05.322885 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.322889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.323034 (+ 145us) server_negotiation.cc:407] Connection header received
0806 18:48:05.323160 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.323163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.323258 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.323367 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:48:05.323537 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.324037 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.324832 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.325060 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.325132 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.325374 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:48:05.325507 (+ 133us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 18:48:05.325705 (+ 198us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54518: token expired
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 18:48:05.325894 25889 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54518: token expired
W20260806 18:48:05.326257 25078 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.245.126:33973)
I20260806 18:48:05.333668 25889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.326630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54530 (local address 127.1.245.126:33973)
0806 18:48:05.326734 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.326737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.326748 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:48:05.326826 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.326828 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.326864 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.326924 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:05.327542 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.328050 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.328862 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.329072 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.329156 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.330584 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:05.330587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:05.330590 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:05.330613 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:05.332070 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:05.332474 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:05.332478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:05.332479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:05.332560 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:05.332879 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:05.332882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:05.332883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:05.333083 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:05.333202 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.333436 (+ 234us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.333543 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:48:05.336318 25204 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54530:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:48:05.339672 25204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:48:05.348155 25386 tablet_service.cc:1467] Processing CreateTablet for tablet fc9cde690333455b8bb3811823d17cb8 (DEFAULT_TABLE table=test-table [id=114723d91ee448878882bfb02ad30b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:05.348858 25386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc9cde690333455b8bb3811823d17cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:05.351850 25319 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Bootstrap starting.
I20260806 18:48:05.352182 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.346853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:36125 (local address 127.0.0.1:52790)
0806 18:48:05.347029 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:05.347043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:05.347125 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:05.348044 (+ 919us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:05.348046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:05.348065 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:05.348172 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.348179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.349389 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.349392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.350583 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.350595 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.351710 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.351714 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.351837 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.351885 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 18:48:05.351950 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:48:05.352038 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260806 18:48:05.352298 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.347041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.1.245.67:36125)
0806 18:48:05.347528 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.347545 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.347571 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:48:05.347656 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.347662 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.347891 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.348043 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:05.348281 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.349276 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.350718 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.351577 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.351692 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.351965 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.352032 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.352148 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":159,"threads_started":1}
I20260806 18:48:05.352826 25319 tablet_bootstrap.cc:654] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:05.353682 25899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.347391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57866 (local address 127.1.245.66:43889)
0806 18:48:05.347942 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.347953 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.347974 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:48:05.348048 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.348053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.348229 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.348403 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:05.348722 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.349758 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.351095 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.351662 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.351765 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.353365 (+ 1600us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.353433 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.353541 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":153,"threads_started":1}
I20260806 18:48:05.354107 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.347121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:43889 (local address 127.0.0.1:57866)
0806 18:48:05.347541 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:05.347554 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:05.347643 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:05.348489 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:05.348492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:05.348513 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:05.348602 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.348608 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.349879 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.349883 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.350977 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.350985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.352062 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.352065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.352153 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.352173 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:48:05.352231 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 18:48:05.353977 (+ 1746us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":115,"threads_started":1}
I20260806 18:48:05.354413 25319 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: No bootstrap required, opened a new log
I20260806 18:48:05.354514 25319 ts_tablet_manager.cc:1397] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260806 18:48:05.354910 25319 raft_consensus.cc:348] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.355000 25319 raft_consensus.cc:374] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:05.355026 25319 raft_consensus.cc:729] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284aa6ffc0eb445a9ae22629f99238d8, State: Initialized, Role: FOLLOWER
I20260806 18:48:05.355140 25319 consensus_queue.cc:260] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.355427 25319 ts_tablet_manager.cc:1428] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 18:48:05.355890 25795 tablet_service.cc:1467] Processing CreateTablet for tablet fc9cde690333455b8bb3811823d17cb8 (DEFAULT_TABLE table=test-table [id=114723d91ee448878882bfb02ad30b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:05.357100 25795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc9cde690333455b8bb3811823d17cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:05.357190 25596 tablet_service.cc:1467] Processing CreateTablet for tablet fc9cde690333455b8bb3811823d17cb8 (DEFAULT_TABLE table=test-table [id=114723d91ee448878882bfb02ad30b1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:05.358253 25596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc9cde690333455b8bb3811823d17cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:05.365769 25901 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Bootstrap starting.
I20260806 18:48:05.366071 25902 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: Bootstrap starting.
I20260806 18:48:05.368706 25901 tablet_bootstrap.cc:654] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:05.368706 25902 tablet_bootstrap.cc:654] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:05.369657 25901 log.cc:824] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:05.369657 25902 log.cc:824] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:05.371863 25902 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: No bootstrap required, opened a new log
I20260806 18:48:05.371872 25901 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: No bootstrap required, opened a new log
I20260806 18:48:05.372045 25901 ts_tablet_manager.cc:1397] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:48:05.372048 25902 ts_tablet_manager.cc:1397] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:48:05.374915 25902 raft_consensus.cc:348] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.374915 25901 raft_consensus.cc:348] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.375147 25902 raft_consensus.cc:374] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:05.375149 25901 raft_consensus.cc:374] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:05.375191 25902 raft_consensus.cc:729] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a07a75d32ef5488c98cd8a75a361c93f, State: Initialized, Role: FOLLOWER
I20260806 18:48:05.375203 25901 raft_consensus.cc:729] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe2fa64900e349db9829f7c73c93d528, State: Initialized, Role: FOLLOWER
I20260806 18:48:05.375604 25901 consensus_queue.cc:260] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.375604 25902 consensus_queue.cc:260] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.376305 25884 heartbeater.cc:503] Master 127.1.245.126:33973 was elected leader, sending a full tablet report...
I20260806 18:48:05.376543 25902 ts_tablet_manager.cc:1428] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 18:48:05.376556 25901 ts_tablet_manager.cc:1428] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260806 18:48:05.377318 25690 heartbeater.cc:503] Master 127.1.245.126:33973 was elected leader, sending a full tablet report...
W20260806 18:48:05.399716 25485 tablet.cc:2369] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:05.527016 25907 raft_consensus.cc:482] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:05.527244 25907 raft_consensus.cc:504] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.528455 25907 leader_election.cc:288] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125), 284aa6ffc0eb445a9ae22629f99238d8 (127.1.245.65:34507)
I20260806 18:48:05.532541 25897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.529020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:43525 (local address 127.1.245.67:36125)
0806 18:48:05.529150 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.529155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.529353 (+ 198us) server_negotiation.cc:407] Connection header received
0806 18:48:05.529464 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.529467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.529514 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.529592 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:05.529934 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.530559 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.531606 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.532005 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.532111 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.532168 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.532338 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.532408 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:48:05.532553 25692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.528756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34507 (local address 127.1.245.66:46413)
0806 18:48:05.528873 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:05.528888 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:05.529001 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:05.529544 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:05.529547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:05.529564 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:05.529706 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.529714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.530559 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.530562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.531443 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.531453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.532134 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.532137 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.532252 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.532272 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:48:05.532331 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 18:48:05.532440 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":24}
I20260806 18:48:05.532553 25909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.528870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:46413 (local address 127.1.245.65:34507)
0806 18:48:05.529275 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.529280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.529298 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:48:05.529360 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.529363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.529428 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.529526 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:05.529853 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.530444 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.531563 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.532015 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.532124 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.532302 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.532345 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.532411 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":97,"threads_started":1}
I20260806 18:48:05.532565 25911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.528756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:36125 (local address 127.1.245.66:43525)
0806 18:48:05.529265 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:05.529280 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:05.529380 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:05.529636 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:05.529639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:05.529659 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:05.529810 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.529817 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.530662 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.530664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.531482 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:05.531492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.532125 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.532128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:05.532219 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.532243 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:48:05.532315 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:48:05.532406 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":72,"thread_start_us":110,"threads_started":1}
I20260806 18:48:05.533249 25419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc9cde690333455b8bb3811823d17cb8" candidate_uuid: "a07a75d32ef5488c98cd8a75a361c93f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284aa6ffc0eb445a9ae22629f99238d8" is_pre_election: true
I20260806 18:48:05.533517 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc9cde690333455b8bb3811823d17cb8" candidate_uuid: "a07a75d32ef5488c98cd8a75a361c93f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2fa64900e349db9829f7c73c93d528" is_pre_election: true
I20260806 18:48:05.533562 25419 raft_consensus.cc:2454] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07a75d32ef5488c98cd8a75a361c93f in term 0.
I20260806 18:48:05.533782 25824 raft_consensus.cc:2454] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a07a75d32ef5488c98cd8a75a361c93f in term 0.
I20260806 18:48:05.534097 25524 leader_election.cc:302] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284aa6ffc0eb445a9ae22629f99238d8, a07a75d32ef5488c98cd8a75a361c93f; no voters:
I20260806 18:48:05.534411 25907 raft_consensus.cc:2781] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:48:05.534503 25907 raft_consensus.cc:482] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:48:05.534554 25907 raft_consensus.cc:3037] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:05.535581 25907 raft_consensus.cc:504] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.535903 25907 leader_election.cc:288] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [CANDIDATE]: Term 1 election: Requested vote from peers fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125), 284aa6ffc0eb445a9ae22629f99238d8 (127.1.245.65:34507)
I20260806 18:48:05.536276 25419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc9cde690333455b8bb3811823d17cb8" candidate_uuid: "a07a75d32ef5488c98cd8a75a361c93f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "284aa6ffc0eb445a9ae22629f99238d8"
I20260806 18:48:05.536401 25419 raft_consensus.cc:3037] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:05.537510 25419 raft_consensus.cc:2454] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07a75d32ef5488c98cd8a75a361c93f in term 1.
I20260806 18:48:05.537807 25524 leader_election.cc:302] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 284aa6ffc0eb445a9ae22629f99238d8, a07a75d32ef5488c98cd8a75a361c93f; no voters:
I20260806 18:48:05.537781 25824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc9cde690333455b8bb3811823d17cb8" candidate_uuid: "a07a75d32ef5488c98cd8a75a361c93f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe2fa64900e349db9829f7c73c93d528"
I20260806 18:48:05.537918 25824 raft_consensus.cc:3037] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:05.538002 25907 raft_consensus.cc:2781] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:05.538229 25907 raft_consensus.cc:686] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 1 LEADER]: Becoming Leader. State: Replica: a07a75d32ef5488c98cd8a75a361c93f, State: Running, Role: LEADER
I20260806 18:48:05.538640 25907 consensus_queue.cc:237] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:05.539302 25824 raft_consensus.cc:2454] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a07a75d32ef5488c98cd8a75a361c93f in term 1.
I20260806 18:48:05.542495 25202 catalog_manager.cc:5495] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f reported cstate change: term changed from 0 to 1, leader changed from <none> to a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66). New cstate: current_term: 1 leader_uuid: "a07a75d32ef5488c98cd8a75a361c93f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } health_report { overall_health: UNKNOWN } } }
W20260806 18:48:05.549372 25887 tablet.cc:2369] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:05.591190 25899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.587848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57878 (local address 127.1.245.66:43889)
0806 18:48:05.587997 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:05.588001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:05.588014 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:48:05.588093 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:05.588095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:05.588152 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:05.588239 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:48:05.588434 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.588929 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.589748 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:05.589976 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:05.590066 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:05.590390 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:48:05.590828 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:48:05.590923 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:05.590970 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 18:48:05.591024 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:48:05.597019 25824 raft_consensus.cc:1261] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 1 FOLLOWER]: Refusing update from remote peer a07a75d32ef5488c98cd8a75a361c93f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:05.597016 25419 raft_consensus.cc:1261] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Refusing update from remote peer a07a75d32ef5488c98cd8a75a361c93f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:05.597671 25912 consensus_queue.cc:1035] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:05.597901 25907 consensus_queue.cc:1035] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:05.613010 25204 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:05.613226 25204 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54530:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:05.615816 25204 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Sending DeleteTablet for 3 replicas of tablet fc9cde690333455b8bb3811823d17cb8
I20260806 18:48:05.616506 25386 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9cde690333455b8bb3811823d17cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:05 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:46376
I20260806 18:48:05.616595 25596 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9cde690333455b8bb3811823d17cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:05 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:57866
I20260806 18:48:05.616595 25795 tablet_service.cc:1514] Processing DeleteTablet for tablet fc9cde690333455b8bb3811823d17cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:05 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:52790
I20260806 18:48:05.616969 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 25263
I20260806 18:48:05.617120 25927 tablet_replica.cc:330] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: stopping tablet replica
I20260806 18:48:05.617127 25930 tablet_replica.cc:330] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: stopping tablet replica
I20260806 18:48:05.617313 25927 raft_consensus.cc:2229] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:05.617480 25930 raft_consensus.cc:2229] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:05.617841 25930 raft_consensus.cc:2258] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:05.618232 25929 tablet_replica.cc:330] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: stopping tablet replica
I20260806 18:48:05.618510 25929 raft_consensus.cc:2229] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:48:05.618862 25929 raft_consensus.cc:2258] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:05.621472 25929 ts_tablet_manager.cc:1905] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:05.622663 25930 ts_tablet_manager.cc:1905] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:05.624267 25929 ts_tablet_manager.cc:1918] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:05.624384 25929 log.cc:1196] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/fc9cde690333455b8bb3811823d17cb8
I20260806 18:48:05.624704 25929 ts_tablet_manager.cc:1939] T fc9cde690333455b8bb3811823d17cb8 P a07a75d32ef5488c98cd8a75a361c93f: Deleting consensus metadata
I20260806 18:48:05.625308 25930 ts_tablet_manager.cc:1918] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:05.625408 25930 log.cc:1196] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/fc9cde690333455b8bb3811823d17cb8
I20260806 18:48:05.625735 25930 ts_tablet_manager.cc:1939] T fc9cde690333455b8bb3811823d17cb8 P fe2fa64900e349db9829f7c73c93d528: Deleting consensus metadata
I20260806 18:48:05.626058 25178 catalog_manager.cc:4841] TS a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889): tablet fc9cde690333455b8bb3811823d17cb8 (table test-table [id=114723d91ee448878882bfb02ad30b1e]) successfully deleted
I20260806 18:48:05.626892 25175 catalog_manager.cc:4841] TS fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125): tablet fc9cde690333455b8bb3811823d17cb8 (table test-table [id=114723d91ee448878882bfb02ad30b1e]) successfully deleted
W20260806 18:48:05.633769 25179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:34507 (error 108)
I20260806 18:48:05.634462 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 25492
I20260806 18:48:05.635071 25898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:05.634702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:34507 (local address 127.0.0.1:46382)
0806 18:48:05.634915 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:05.634981 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:34507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260806 18:48:05.637724 25179 catalog_manager.cc:4584] TS 284aa6ffc0eb445a9ae22629f99238d8 (127.1.245.65:34507): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fc9cde690333455b8bb3811823d17cb8: Network error: Client connection negotiation failed: client connection to 127.1.245.65:34507: connect: Connection refused (error 111)
I20260806 18:48:05.651150 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 25693
I20260806 18:48:05.662819 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 25145
I20260806 18:48:25.675050 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33973
--webserver_interface=127.1.245.126
--webserver_port=43973
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:33973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:25.788478 25933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:25.788781 25933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:25.788842 25933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:25.792927 25933 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 18:48:25.792990 25933 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 18:48:25.793011 25933 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 18:48:25.793030 25933 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:25.793052 25933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:25.793087 25933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:25.793118 25933 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:25.793144 25933 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:25.798763 25933 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:33973
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:33973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=43973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:25.799978 25933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:25.801066 25933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:25.810065 25944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:25.810111 25949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:25.810122 25943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:25.810204 25933 server_base.cc:1034] running on GCE node
I20260806 18:48:25.811025 25933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:25.812388 25933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:25.813566 25933 hybrid_clock.cc:648] HybridClock initialized: now 1786042105813549 us; error 32 us; skew 500 ppm
Aug 06 18:48:25 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042105, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:25.820067 25933 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:48:25.821085 25933 webserver.cc:460] Webserver started at http://127.1.245.126:43973/ using document root <none> and password file <none>
I20260806 18:48:25.821722 25933 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:25.821791 25933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:25.825882 25933 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:48:25.828280 25960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:25.829571 25933 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:48:25.829690 25933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "3ac19ea0977f4861bf1e52218e8943a7"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "b77c7ccc2c090aa7e6de6183d86adc6a"
server_key_iv: "360c6fc625ece6a29c60d426b44119b2"
server_key_version: "encryptionkey@0"
I20260806 18:48:25.830406 25933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:25.840551 25933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:25.843608 25933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:25.843792 25933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:25.850893 25933 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:33973
I20260806 18:48:25.851167 26039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:33973 every 8 connection(s)
I20260806 18:48:25.852037 25933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:25.852286 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 25933
I20260806 18:48:25.852778 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:34507
--local_ip_for_outbound_sockets=127.1.245.65
--tserver_master_addrs=127.1.245.126:33973
--webserver_port=41373
--webserver_interface=127.1.245.65
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:48:25.857062 26040 sys_catalog.cc:263] Verifying existing consensus state
I20260806 18:48:25.858868 26040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Bootstrap starting.
I20260806 18:48:25.876585 26040 log.cc:824] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:25.887902 26040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:48:25.888264 26040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Bootstrap complete.
I20260806 18:48:25.892194 26040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:48:25.892591 26040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ac19ea0977f4861bf1e52218e8943a7, State: Initialized, Role: FOLLOWER
I20260806 18:48:25.893064 26040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:48:25.893193 26040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:25.893262 26040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:25.893356 26040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 2 FOLLOWER]: Advancing to term 3
I20260806 18:48:25.895296 26040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:48:25.895614 26040 leader_election.cc:302] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ac19ea0977f4861bf1e52218e8943a7; no voters:
I20260806 18:48:25.895937 26040 leader_election.cc:288] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 18:48:25.896049 26044 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 18:48:25.896248 26044 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [term 3 LEADER]: Becoming Leader. State: Replica: 3ac19ea0977f4861bf1e52218e8943a7, State: Running, Role: LEADER
I20260806 18:48:25.896546 26044 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } }
I20260806 18:48:25.897245 26040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:25.898018 26048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ac19ea0977f4861bf1e52218e8943a7. Latest consensus state: current_term: 3 leader_uuid: "3ac19ea0977f4861bf1e52218e8943a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } } }
I20260806 18:48:25.898059 26047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3ac19ea0977f4861bf1e52218e8943a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ac19ea0977f4861bf1e52218e8943a7" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 33973 } } }
I20260806 18:48:25.898152 26048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:25.898177 26047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:25.898541 26052 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:25.903708 26052 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:25.904461 26052 catalog_manager.cc:1259] Loaded cluster ID: ec4094cab337420896a619c4b681e44c
I20260806 18:48:25.904524 26052 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:25.907090 26052 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:25.907871 26052 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Loaded TSK: 0
I20260806 18:48:25.907980 26052 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3ac19ea0977f4861bf1e52218e8943a7: Loaded TSK: 1
F20260806 18:48:25.908772 26052 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7febd0c809ad google::LogMessage::Fail() at ??:0
@ 0x7febd0c8388a google::LogMessage::SendToLog() at ??:0
@ 0x7febd0c804de google::LogMessage::Flush() at ??:0
@ 0x7febd0c81a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7febd394418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7febd393cf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7febdb5e57bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7febdb5bf99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7febdb5c0178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7febdb601df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7febdb62082b std::function<>::operator()() at ??:0
@ 0x7febdb5bfb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7febdb5c0f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7febdb5bd579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7febdb601640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7febd2ed23d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7febd2ed2c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7febd2ed43f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7febd2ec3512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7febd45076db start_thread at ??:0
@ 0x7febcf77e61f clone at ??:0
W20260806 18:48:25.980516 26042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:25.980762 26042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:25.980809 26042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:25.985110 26042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:25.985167 26042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:25.985235 26042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:48:25.990983 26042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:34507
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=41373
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:25.992051 26042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:25.993052 26042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:26.003159 26102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:26.003237 26098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:26.003189 26105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:26.003690 26042 server_base.cc:1034] running on GCE node
I20260806 18:48:26.004117 26042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:26.004774 26042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:26.005949 26042 hybrid_clock.cc:648] HybridClock initialized: now 1786042106005918 us; error 49 us; skew 500 ppm
Aug 06 18:48:26 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042106, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:26.009409 26042 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:48:26.010771 26042 webserver.cc:460] Webserver started at http://127.1.245.65:41373/ using document root <none> and password file <none>
I20260806 18:48:26.011602 26042 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:26.011667 26042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:26.017144 26042 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:48:26.019781 26120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:26.021140 26042 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:48:26.021268 26042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "284aa6ffc0eb445a9ae22629f99238d8"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "e5b9964aa3e029703c981b9f275aa96a"
server_key_iv: "26e8e9edd21e9f59eb2b2d9c0c0094f9"
server_key_version: "encryptionkey@0"
I20260806 18:48:26.021858 26042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:26.033382 26042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:26.036689 26042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:26.036875 26042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:26.037489 26042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:26.038677 26130 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 18:48:26.044310 26042 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 18:48:26.044401 26042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260806 18:48:26.044480 26042 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 18:48:26.046283 26042 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 18:48:26.046366 26042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260806 18:48:26.046717 26130 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Bootstrap starting.
I20260806 18:48:26.054908 26042 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:34507
I20260806 18:48:26.055863 26284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:34507 every 8 connection(s)
I20260806 18:48:26.056059 26042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:48:26.060459 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 26042
I20260806 18:48:26.060921 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:43889
--local_ip_for_outbound_sockets=127.1.245.66
--tserver_master_addrs=127.1.245.126:33973
--webserver_port=41489
--webserver_interface=127.1.245.66
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:48:26.070554 26130 log.cc:824] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:26.077924 26130 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 18:48:26.078380 26130 tablet_bootstrap.cc:492] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Bootstrap complete.
I20260806 18:48:26.079448 26130 ts_tablet_manager.cc:1397] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Time spent bootstrapping tablet: real 0.033s user 0.019s sys 0.011s
I20260806 18:48:26.081931 26130 raft_consensus.cc:348] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:26.082396 26130 raft_consensus.cc:729] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 284aa6ffc0eb445a9ae22629f99238d8, State: Initialized, Role: FOLLOWER
I20260806 18:48:26.082880 26130 consensus_queue.cc:260] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:26.083876 26130 ts_tablet_manager.cc:1428] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260806 18:48:26.178572 26289 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:48:26.057987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:54309)
0806 18:48:26.058543 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.058596 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:26.059548 (+ 952us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:26.177632 (+118084us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":125,"threads_started":1}
W20260806 18:48:26.179086 26285 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:33973 (0 consecutive failures): Network error: Failed to ping master at 127.1.245.126:33973: Client connection negotiation failed: client connection to 127.1.245.126:33973: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 18:48:26.179818 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:26.179564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:49371)
0806 18:48:26.179690 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.179762 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:48:26.180486 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:26.180278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:37273)
0806 18:48:26.180392 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.180438 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:48:26.180636 26285 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:48:26.208002 26290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:26.208294 26290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:26.208359 26290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:26.212514 26290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:26.212585 26290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:26.212666 26290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:48:26.218621 26290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:43889
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=41489
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:26.219756 26290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:26.220825 26290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:26.230887 26300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:26.230897 26301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:26.231109 26305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:26.231122 26290 server_base.cc:1034] running on GCE node
I20260806 18:48:26.231722 26290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:26.232376 26290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:26.233559 26290 hybrid_clock.cc:648] HybridClock initialized: now 1786042106233537 us; error 32 us; skew 500 ppm
Aug 06 18:48:26 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042106, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:26.236735 26290 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:48:26.237666 26290 webserver.cc:460] Webserver started at http://127.1.245.66:41489/ using document root <none> and password file <none>
I20260806 18:48:26.238263 26290 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:26.238370 26290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:26.242410 26290 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:48:26.244577 26315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:26.245847 26290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:48:26.245965 26290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "a07a75d32ef5488c98cd8a75a361c93f"
format_stamp: "Formatted at 2026-08-06 18:47:43 on dist-test-slave-fgb3"
server_key: "3c4f2a11091121b156c989582bc5dd66"
server_key_iv: "5630aeb4d9b134f56c1d095eab519d6b"
server_key_version: "encryptionkey@0"
I20260806 18:48:26.246578 26290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:26.264693 26290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:26.267815 26290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:26.268019 26290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:26.268682 26290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:26.269568 26290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:26.269619 26290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:26.269675 26290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:26.269703 26290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:26.279359 26290 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:43889
I20260806 18:48:26.279399 26482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:43889 every 8 connection(s)
I20260806 18:48:26.280467 26290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:48:26.282136 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 26290
I20260806 18:48:26.282661 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:36125
--local_ip_for_outbound_sockets=127.1.245.67
--tserver_master_addrs=127.1.245.126:33973
--webserver_port=44093
--webserver_interface=127.1.245.67
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:40443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:48:26.284669 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:26.283011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:55289)
0806 18:48:26.283777 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.283892 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":157,"threads_started":1}
W20260806 18:48:26.285228 26485 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:33973 (0 consecutive failures): Network error: Failed to ping master at 127.1.245.126:33973: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
I20260806 18:48:26.285902 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:26.285656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:48789)
0806 18:48:26.285782 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.285831 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260806 18:48:26.286500 26488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:26.286293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:50151)
0806 18:48:26.286410 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.286452 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260806 18:48:26.286644 26485 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 18:48:26.396838 26489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:26.397105 26489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:26.397148 26489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:26.401108 26489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:26.401158 26489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:26.401212 26489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:48:26.406231 26489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:40443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:36125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=44093
--tserver_master_addrs=127.1.245.126:33973
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:26.407243 26489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:26.408188 26489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:26.417701 26505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:26.417749 26501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:26.417814 26489 server_base.cc:1034] running on GCE node
W20260806 18:48:26.417752 26502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:26.418339 26489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:26.418972 26489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:26.420126 26489 hybrid_clock.cc:648] HybridClock initialized: now 1786042106420108 us; error 31 us; skew 500 ppm
Aug 06 18:48:26 dist-test-slave-fgb3 krb5kdc[24293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042106, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:26.422741 26489 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:48:26.423590 26489 webserver.cc:460] Webserver started at http://127.1.245.67:44093/ using document root <none> and password file <none>
I20260806 18:48:26.424129 26489 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:26.424196 26489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:26.428125 26489 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:48:26.430123 26517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:26.431337 26489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:48:26.431465 26489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "fe2fa64900e349db9829f7c73c93d528"
format_stamp: "Formatted at 2026-08-06 18:47:44 on dist-test-slave-fgb3"
server_key: "abef1e8162da6606d33e10ab615be191"
server_key_iv: "0ec469ba84aa7c3734ca90bd6173f28e"
server_key_version: "encryptionkey@0"
I20260806 18:48:26.431972 26489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:26.455466 26489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:26.458357 26489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:26.458545 26489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:26.459152 26489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:26.460001 26489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:26.460048 26489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:26.460103 26489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:26.460131 26489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:26.468711 26489 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:36125
I20260806 18:48:26.468743 26691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:36125 every 8 connection(s)
I20260806 18:48:26.469728 26489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:48:26.470271 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 26489
I20260806 18:48:26.473409 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:26.471961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:43429)
0806 18:48:26.472598 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.472702 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":105,"threads_started":1}
W20260806 18:48:26.473767 26692 heartbeater.cc:644] Failed to heartbeat to 127.1.245.126:33973 (0 consecutive failures): Network error: Failed to ping master at 127.1.245.126:33973: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
I20260806 18:48:26.474476 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:26.474112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:57211)
0806 18:48:26.474261 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.474359 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 18:48:26.475019 26698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:26.474837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:49929)
0806 18:48:26.474926 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:26.474970 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:48:26.475176 26692 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 18:48:27.181843 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:27.181343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:46319)
0806 18:48:27.181645 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:27.181738 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":91,"threads_started":1}
I20260806 18:48:27.287678 26701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:27.287240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:50435)
0806 18:48:27.287504 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:27.287568 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":70,"threads_started":1}
I20260806 18:48:27.476308 26702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:27.475789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:51145)
0806 18:48:27.476102 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:27.476176 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":75,"threads_started":1}
I20260806 18:48:27.855655 26704 raft_consensus.cc:482] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:27.855937 26704 raft_consensus.cc:504] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:27.857053 26704 leader_election.cc:288] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125), a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889)
Aug 06 18:48:27 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042106, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.66@KRBTEST.COM
Aug 06 18:48:27 dist-test-slave-fgb3 krb5kdc[24293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042106, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.67@KRBTEST.COM
I20260806 18:48:27.870850 26708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:27.857364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:43889 (local address 127.1.245.65:36165)
0806 18:48:27.857771 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:27.857787 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:27.857898 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:27.859551 (+ 1653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:27.859557 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:27.860108 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:27.860644 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:27.860662 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:27.862039 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:27.862042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:27.863003 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:27.863014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:27.863837 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:27.863841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:27.864060 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:27.864671 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:27.864688 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:27.866346 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:27.868386 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:27.868392 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:27.868406 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:27.868652 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:27.868994 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:27.868998 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:27.869000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:27.869146 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:27.869796 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:27.869803 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:27.870063 (+ 260us) client_negotiation.cc:769] Sending connection context
0806 18:48:27.870237 (+ 174us) client_negotiation.cc:240] Negotiation successful
0806 18:48:27.870617 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":125,"spinlock_wait_cycles":16640,"thread_start_us":104,"threads_started":1}
I20260806 18:48:27.870864 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:27.857559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:36165 (local address 127.1.245.66:43889)
0806 18:48:27.857934 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:27.857964 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:27.858014 (+ 50us) server_negotiation.cc:407] Connection header received
0806 18:48:27.858809 (+ 795us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:27.858816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:27.859165 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:27.859459 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:27.860857 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:27.861878 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:27.863124 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:27.863713 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:27.863876 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:27.866570 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:27.866576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:27.866589 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:27.866619 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:27.868242 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:27.868765 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:27.868770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:27.868774 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:27.868892 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:27.869243 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:27.869246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:27.869247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:27.869670 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:27.869796 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:27.870421 (+ 625us) server_negotiation.cc:299] Negotiation successful
0806 18:48:27.870692 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":83,"threads_started":1}
I20260806 18:48:27.870893 26710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:27.857485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:49877 (local address 127.1.245.67:36125)
0806 18:48:27.857845 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:27.857873 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:27.858059 (+ 186us) server_negotiation.cc:407] Connection header received
0806 18:48:27.858822 (+ 763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:27.858828 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:27.859165 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:27.859458 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:27.860898 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:27.861967 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:27.863144 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:27.863725 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:27.863877 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:27.866839 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:27.866843 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:27.866854 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:27.866903 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:27.868541 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:27.868950 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:27.868956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:27.868962 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:27.869076 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:27.869337 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:27.869340 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:27.869342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:27.869716 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:27.869833 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:27.870415 (+ 582us) server_negotiation.cc:299] Negotiation successful
0806 18:48:27.870683 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":78,"threads_started":1}
I20260806 18:48:27.870885 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:27.857362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:36125 (local address 127.1.245.65:49877)
0806 18:48:27.857901 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:27.857931 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:27.858081 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:27.859551 (+ 1470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:27.859557 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:27.860108 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:27.860643 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:27.860661 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:27.862101 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:27.862103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:27.863008 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:27.863021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:27.863845 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:27.863849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:27.864060 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:27.864713 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:27.864726 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:27.866664 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:27.868653 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:27.868656 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:27.868659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:27.868844 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:27.869170 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:27.869173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:27.869175 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:27.869256 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:27.869823 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:27.869826 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:27.870062 (+ 236us) client_negotiation.cc:769] Sending connection context
0806 18:48:27.870237 (+ 175us) client_negotiation.cc:240] Negotiation successful
0806 18:48:27.870698 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"mutex_wait_us":407,"spinlock_wait_cycles":13312,"thread_start_us":54,"threads_started":1}
W20260806 18:48:27.873450 26123 leader_election.cc:341] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889): Not found: Tablet not found: fc9cde690333455b8bb3811823d17cb8
W20260806 18:48:27.874114 26121 leader_election.cc:341] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125): Not found: Tablet not found: fc9cde690333455b8bb3811823d17cb8
I20260806 18:48:27.874207 26121 leader_election.cc:302] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 284aa6ffc0eb445a9ae22629f99238d8; no voters: a07a75d32ef5488c98cd8a75a361c93f, fe2fa64900e349db9829f7c73c93d528
I20260806 18:48:27.874523 26704 raft_consensus.cc:2726] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:48:28.183013 26709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:28.182701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:45793)
0806 18:48:28.182849 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:28.182915 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 18:48:28.289085 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:28.288607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:52229)
0806 18:48:28.288892 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:28.288960 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260806 18:48:28.477541 26716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:28.477108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:52355)
0806 18:48:28.477382 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:28.477443 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260806 18:48:29.184433 26717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:29.183925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:37223)
0806 18:48:29.184252 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:29.184326 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":83,"threads_started":1}
I20260806 18:48:29.290522 26719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:29.289962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:33717)
0806 18:48:29.290336 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:29.290412 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":83,"threads_started":1}
I20260806 18:48:29.478834 26720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:29.478378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:42771)
0806 18:48:29.478642 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:29.478704 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":79,"threads_started":1}
I20260806 18:48:29.938053 26723 raft_consensus.cc:482] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:29.938195 26723 raft_consensus.cc:504] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:29.938663 26723 leader_election.cc:288] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125), a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889)
W20260806 18:48:29.939620 26121 leader_election.cc:341] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125): Not found: Tablet not found: fc9cde690333455b8bb3811823d17cb8
W20260806 18:48:29.939757 26123 leader_election.cc:341] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889): Not found: Tablet not found: fc9cde690333455b8bb3811823d17cb8
I20260806 18:48:29.939873 26123 leader_election.cc:302] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 284aa6ffc0eb445a9ae22629f99238d8; no voters: a07a75d32ef5488c98cd8a75a361c93f, fe2fa64900e349db9829f7c73c93d528
I20260806 18:48:29.940019 26723 raft_consensus.cc:2726] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:48:30.185869 26724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:30.185352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:37725)
0806 18:48:30.185656 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:30.185726 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":86,"threads_started":1}
I20260806 18:48:30.291889 26725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:30.291381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:48981)
0806 18:48:30.291702 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:30.291772 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":110,"threads_started":1}
I20260806 18:48:30.480032 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:30.479577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:52719)
0806 18:48:30.479878 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:30.479937 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":96,"threads_started":1}
I20260806 18:48:31.187448 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:31.186970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:50073)
0806 18:48:31.187285 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:31.187345 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":85,"threads_started":1}
I20260806 18:48:31.293309 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:31.292829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:54859)
0806 18:48:31.293123 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:31.293196 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":88,"threads_started":1}
I20260806 18:48:31.481244 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:31.480801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:43443)
0806 18:48:31.481076 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:31.481143 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":80,"threads_started":1}
I20260806 18:48:32.188797 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:32.188336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:51605)
0806 18:48:32.188612 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:32.188676 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260806 18:48:32.294623 26733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:32.294126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:60073)
0806 18:48:32.294455 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:32.294521 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":90,"threads_started":1}
I20260806 18:48:32.482411 26735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:32.481952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:40427)
0806 18:48:32.482213 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:32.482269 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":99,"threads_started":1}
I20260806 18:48:32.957446 26737 raft_consensus.cc:482] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:32.957581 26737 raft_consensus.cc:504] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:32.958019 26737 leader_election.cc:288] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125), a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889)
W20260806 18:48:32.958824 26123 leader_election.cc:341] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889): Not found: Tablet not found: fc9cde690333455b8bb3811823d17cb8
W20260806 18:48:32.958930 26121 leader_election.cc:341] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125): Not found: Tablet not found: fc9cde690333455b8bb3811823d17cb8
I20260806 18:48:32.959013 26121 leader_election.cc:302] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 284aa6ffc0eb445a9ae22629f99238d8; no voters: a07a75d32ef5488c98cd8a75a361c93f, fe2fa64900e349db9829f7c73c93d528
I20260806 18:48:32.959193 26737 raft_consensus.cc:2726] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:48:33.190110 26739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:33.189618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:38979)
0806 18:48:33.189937 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:33.190004 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":105,"threads_started":1}
I20260806 18:48:33.295855 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:33.295375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:58985)
0806 18:48:33.295687 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:33.295743 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":74,"threads_started":1}
I20260806 18:48:33.483677 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:33.483165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:53011)
0806 18:48:33.483513 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:33.483574 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":79,"threads_started":1}
I20260806 18:48:34.191517 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:34.190998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:46765)
0806 18:48:34.191345 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:34.191412 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":99,"threads_started":1}
I20260806 18:48:34.296990 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:34.296567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:60845)
0806 18:48:34.296833 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:34.296889 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":74,"threads_started":1}
I20260806 18:48:34.484985 26747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:34.484472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:50345)
0806 18:48:34.484818 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:34.484885 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
I20260806 18:48:35.192867 26749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:35.192401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:42695)
0806 18:48:35.192682 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:35.192750 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":94,"threads_started":1}
I20260806 18:48:35.298300 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:35.297778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:40379)
0806 18:48:35.298100 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:35.298159 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":90,"threads_started":1}
I20260806 18:48:35.486480 26751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:35.485906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:40763)
0806 18:48:35.486219 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:35.486363 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":100,"threads_started":1}
I20260806 18:48:36.194542 26752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:36.193945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:54163)
0806 18:48:36.194299 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:36.194433 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":87,"threads_started":1}
I20260806 18:48:36.299820 26753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:36.299337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:44773)
0806 18:48:36.299648 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:36.299716 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":85,"threads_started":1}
I20260806 18:48:36.303663 26754 raft_consensus.cc:482] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:36.303802 26754 raft_consensus.cc:504] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe2fa64900e349db9829f7c73c93d528" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 36125 } } peers { permanent_uuid: "a07a75d32ef5488c98cd8a75a361c93f" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 43889 } } peers { permanent_uuid: "284aa6ffc0eb445a9ae22629f99238d8" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 34507 } }
I20260806 18:48:36.304231 26754 leader_election.cc:288] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125), a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889)
W20260806 18:48:36.304975 26123 leader_election.cc:341] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a07a75d32ef5488c98cd8a75a361c93f (127.1.245.66:43889): Not found: Tablet not found: fc9cde690333455b8bb3811823d17cb8
W20260806 18:48:36.305071 26121 leader_election.cc:341] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fe2fa64900e349db9829f7c73c93d528 (127.1.245.67:36125): Not found: Tablet not found: fc9cde690333455b8bb3811823d17cb8
I20260806 18:48:36.305167 26121 leader_election.cc:302] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 284aa6ffc0eb445a9ae22629f99238d8; no voters: a07a75d32ef5488c98cd8a75a361c93f, fe2fa64900e349db9829f7c73c93d528
I20260806 18:48:36.305349 26754 raft_consensus.cc:2726] T fc9cde690333455b8bb3811823d17cb8 P 284aa6ffc0eb445a9ae22629f99238d8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 18:48:36.487993 26758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:36.487486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:53621)
0806 18:48:36.487820 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:36.487890 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":94,"threads_started":1}
I20260806 18:48:37.195911 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:37.195449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:41295)
0806 18:48:37.195731 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:37.195801 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
I20260806 18:48:37.301101 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:37.300662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:52831)
0806 18:48:37.300933 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:37.300998 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":93,"threads_started":1}
I20260806 18:48:37.489398 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:37.488846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:33089)
0806 18:48:37.489177 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:37.489293 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":81,"threads_started":1}
I20260806 18:48:38.197310 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:38.196819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:39107)
0806 18:48:38.197124 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:38.197194 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":83,"threads_started":1}
I20260806 18:48:38.302395 26766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:38.301895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:42699)
0806 18:48:38.302173 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:38.302241 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":99,"threads_started":1}
I20260806 18:48:38.490785 26767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:38.490250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:60511)
0806 18:48:38.490614 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:38.490679 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":151,"threads_started":1}
I20260806 18:48:39.198547 26769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:39.198116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:59457)
0806 18:48:39.198385 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:39.198443 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":82,"threads_started":1}
I20260806 18:48:39.303675 26770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:39.303237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:51217)
0806 18:48:39.303511 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:39.303574 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260806 18:48:39.492089 26772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:39.491609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:42991)
0806 18:48:39.491902 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:39.491970 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":86,"threads_started":1}
I20260806 18:48:40.199743 26773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:40.199334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:36767)
0806 18:48:40.199583 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:40.199642 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":80,"threads_started":1}
I20260806 18:48:40.305142 26774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:40.304635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:44119)
0806 18:48:40.304974 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:40.305040 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":111,"threads_started":1}
I20260806 18:48:40.493525 26775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:40.492993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:38811)
0806 18:48:40.493356 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:40.493421 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":118,"threads_started":1}
I20260806 18:48:41.201284 26776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:41.200845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.65:45797)
0806 18:48:41.201120 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:41.201181 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
I20260806 18:48:41.306488 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:41.306014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.66:41057)
0806 18:48:41.306295 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:41.306385 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260806 18:48:41.472419 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 26042
I20260806 18:48:41.495044 26779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:41.494573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:33973 (local address 127.1.245.67:54679)
0806 18:48:41.494869 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:41.494941 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.126:33973: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":80,"threads_started":1}
I20260806 18:48:41.496048 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 26290
I20260806 18:48:41.507659 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 26489
2026-08-06T18:48:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59607 ms)
[----------] 30 tests from SecurityITest (247206 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:41 dist-test-slave-fgb3 krb5kdc[26783](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:41 dist-test-slave-fgb3 krb5kdc[26783](info): set up 2 sockets
Aug 06 18:48:41 dist-test-slave-fgb3 krb5kdc[26783](info): commencing operation
krb5kdc: starting...
W20260806 18:48:42.644790 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:48:42 dist-test-slave-fgb3 krb5kdc[26783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:48:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:48:42.771435 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:42009
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:42009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:42.884963 26800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:42.885236 26800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:42.885275 26800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:42.889293 26800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:42.889357 26800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:42.889374 26800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:42.889390 26800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:42.889405 26800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:42.894655 26800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:42009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:42009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:42.895704 26800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:42.896733 26800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:48:42.905699 26800 server_base.cc:1034] running on GCE node
W20260806 18:48:42.905740 26808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:42.905720 26809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:42.905720 26812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:42.906606 26800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:42.907722 26800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:42.908903 26800 hybrid_clock.cc:648] HybridClock initialized: now 1786042122908887 us; error 27 us; skew 500 ppm
Aug 06 18:48:42 dist-test-slave-fgb3 krb5kdc[26783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042122, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:42.911752 26800 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:48:42.912715 26800 webserver.cc:460] Webserver started at http://127.1.245.126:33815/ using document root <none> and password file <none>
I20260806 18:48:42.913275 26800 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:42.913347 26800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:42.913627 26800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:42.915740 26800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "be077f3b7ba7499eb3e5493380843028"
format_stamp: "Formatted at 2026-08-06 18:48:42 on dist-test-slave-fgb3"
server_key: "085fad8f67f70062f302e4ade453c6b9"
server_key_iv: "61fc9de19168959775d5296b67dadbcd"
server_key_version: "encryptionkey@0"
I20260806 18:48:42.916364 26800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "be077f3b7ba7499eb3e5493380843028"
format_stamp: "Formatted at 2026-08-06 18:48:42 on dist-test-slave-fgb3"
server_key: "085fad8f67f70062f302e4ade453c6b9"
server_key_iv: "61fc9de19168959775d5296b67dadbcd"
server_key_version: "encryptionkey@0"
I20260806 18:48:42.920379 26800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:48:42.922786 26825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:42.923988 26800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:42.924103 26800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "be077f3b7ba7499eb3e5493380843028"
format_stamp: "Formatted at 2026-08-06 18:48:42 on dist-test-slave-fgb3"
server_key: "085fad8f67f70062f302e4ade453c6b9"
server_key_iv: "61fc9de19168959775d5296b67dadbcd"
server_key_version: "encryptionkey@0"
I20260806 18:48:42.924202 26800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:42.938643 26800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:42.941414 26800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:42.941593 26800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:42.948776 26800 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:42009
I20260806 18:48:42.948807 26895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:42009 every 8 connection(s)
I20260806 18:48:42.949738 26800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:42.952553 26897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:42.958154 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 26800
I20260806 18:48:42.958288 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:42.958303 26897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028: Bootstrap starting.
I20260806 18:48:42.958633 2005 external_mini_cluster.cc:1428] Setting key 227587a54ddd2a48d928ce87ce79ec93
I20260806 18:48:42.960948 26897 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:42.961671 26897 log.cc:824] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:42.963500 26897 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028: No bootstrap required, opened a new log
Aug 06 18:48:42 dist-test-slave-fgb3 krb5kdc[26783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:42.967259 26897 raft_consensus.cc:348] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be077f3b7ba7499eb3e5493380843028" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42009 } }
I20260806 18:48:42.967538 26897 raft_consensus.cc:374] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:42.967602 26897 raft_consensus.cc:729] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be077f3b7ba7499eb3e5493380843028, State: Initialized, Role: FOLLOWER
I20260806 18:48:42.968071 26897 consensus_queue.cc:260] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be077f3b7ba7499eb3e5493380843028" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42009 } }
I20260806 18:48:42.968235 26897 raft_consensus.cc:388] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:42.968304 26897 raft_consensus.cc:482] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:42.968410 26897 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:42.969322 26897 raft_consensus.cc:504] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be077f3b7ba7499eb3e5493380843028" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42009 } }
I20260806 18:48:42.969657 26897 leader_election.cc:302] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be077f3b7ba7499eb3e5493380843028; no voters:
I20260806 18:48:42.970028 26897 leader_election.cc:288] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:42.970096 26902 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:42.970333 26902 raft_consensus.cc:686] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [term 1 LEADER]: Becoming Leader. State: Replica: be077f3b7ba7499eb3e5493380843028, State: Running, Role: LEADER
I20260806 18:48:42.970701 26902 consensus_queue.cc:237] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be077f3b7ba7499eb3e5493380843028" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42009 } }
I20260806 18:48:42.971288 26897 sys_catalog.cc:564] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:42.973062 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:42.960057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.1.245.126:42009)
0806 18:48:42.960497 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:42.960512 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:42.960546 (+ 34us) server_negotiation.cc:407] Connection header received
0806 18:48:42.961193 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:42.961211 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:42.961543 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:42.961837 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:42.962651 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:42.963629 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:42.964453 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:42.964927 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:42.965069 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:42.967740 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:42.967745 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:42.967758 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:42.967785 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:42.969491 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:42.969938 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:42.969943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:42.969947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:42.970091 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:42.970506 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:42.970509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:42.970511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:42.970998 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:42.971190 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:42.971499 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 18:48:42.971797 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":153,"threads_started":1}
I20260806 18:48:42.973163 26904 sys_catalog.cc:455] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be077f3b7ba7499eb3e5493380843028. Latest consensus state: current_term: 1 leader_uuid: "be077f3b7ba7499eb3e5493380843028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be077f3b7ba7499eb3e5493380843028" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42009 } } }
I20260806 18:48:42.973285 26904 sys_catalog.cc:458] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:42.973393 26903 sys_catalog.cc:455] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be077f3b7ba7499eb3e5493380843028" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be077f3b7ba7499eb3e5493380843028" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42009 } } }
I20260806 18:48:42.973484 26903 sys_catalog.cc:458] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:42.973652 26913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:42.977579 26913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:42.983078 26913 catalog_manager.cc:1347] Generated new cluster ID: 94bc50242a6f420a84872de8934832cb
I20260806 18:48:42.983151 26913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:43.000736 26913 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:43.002102 26913 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:43.018102 26913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028: Generated new TSK 0
I20260806 18:48:43.019089 26913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:48:43.052858 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42009
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:43.166240 26932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:43.166586 26932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:43.166632 26932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:43.170768 26932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:43.170840 26932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:43.170902 26932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:48:43.176468 26932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:43.177590 26932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:43.178707 26932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:43.188305 26939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:43.188345 26941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:43.188369 26948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:43.188515 26932 server_base.cc:1034] running on GCE node
I20260806 18:48:43.188936 26932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:43.189543 26932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:43.190697 26932 hybrid_clock.cc:648] HybridClock initialized: now 1786042123190672 us; error 40 us; skew 500 ppm
Aug 06 18:48:43 dist-test-slave-fgb3 krb5kdc[26783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042123, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:43.193476 26932 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:48:43.194378 26932 webserver.cc:460] Webserver started at http://127.1.245.65:33641/ using document root <none> and password file <none>
I20260806 18:48:43.194928 26932 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:43.194993 26932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:43.195165 26932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:43.197173 26932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "08bc585aa8b642f4b6e4c87ad0117d3d"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "26555c0cb94b3d83e8bcf3a8fe1e8978"
server_key_iv: "7bdc6417cc380db927e83a6e1a23e805"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.197698 26932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "08bc585aa8b642f4b6e4c87ad0117d3d"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "26555c0cb94b3d83e8bcf3a8fe1e8978"
server_key_iv: "7bdc6417cc380db927e83a6e1a23e805"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.201417 26932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:48:43.204030 26967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.205284 26932 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:48:43.205404 26932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "08bc585aa8b642f4b6e4c87ad0117d3d"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "26555c0cb94b3d83e8bcf3a8fe1e8978"
server_key_iv: "7bdc6417cc380db927e83a6e1a23e805"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.205497 26932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:43.223865 26932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:43.227072 26932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:43.227331 26932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:43.227938 26932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:43.228813 26932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:43.228860 26932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.228904 26932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:43.228933 26932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.237337 26932 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:45685
I20260806 18:48:43.237393 27137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:45685 every 8 connection(s)
I20260806 18:48:43.238224 26932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:48:43.239656 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 26932
I20260806 18:48:43.239781 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:48:43.240103 2005 external_mini_cluster.cc:1428] Setting key 0c7f7626936117a9c296d982d434a352
Aug 06 18:48:43 dist-test-slave-fgb3 krb5kdc[26783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042123, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:48:43.251788 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.240132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:34325 (local address 127.1.245.126:42009)
0806 18:48:43.240277 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.240281 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.241223 (+ 942us) server_negotiation.cc:407] Connection header received
0806 18:48:43.242584 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.242587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.242646 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.242718 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:43.243932 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.244464 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.245619 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.245985 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.246126 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.248451 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:43.248456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:43.248459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:43.248493 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:43.249936 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:43.250483 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:43.250487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:43.250489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:43.250566 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:43.250876 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:43.250879 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:43.250880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:43.251049 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:43.251118 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.251543 (+ 425us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.251655 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:48:43.252686 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.240572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:42009 (local address 127.1.245.65:34325)
0806 18:48:43.241067 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:43.241121 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:43.242365 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:43.242910 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:43.242917 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:43.243265 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:43.243782 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.243795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.244582 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.244586 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.245467 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.245479 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.246145 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.246148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.246300 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.246879 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:43.246896 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:43.248280 (+ 1384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:43.250039 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:43.250044 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:43.250055 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:43.250392 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:43.250652 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:43.250655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:43.250656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:43.250792 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:43.251119 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:43.251123 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:43.251356 (+ 233us) client_negotiation.cc:769] Sending connection context
0806 18:48:43.251502 (+ 146us) client_negotiation.cc:240] Negotiation successful
0806 18:48:43.251794 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":125,"threads_started":1}
I20260806 18:48:43.253998 27139 heartbeater.cc:347] Connected to a master server at 127.1.245.126:42009
I20260806 18:48:43.254280 27139 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:43.254889 27139 heartbeater.cc:511] Master 127.1.245.126:42009 requested a full tablet report, sending...
I20260806 18:48:43.256670 26850 ts_manager.cc:194] Registered new tserver with Master: 08bc585aa8b642f4b6e4c87ad0117d3d (127.1.245.65:45685)
I20260806 18:48:43.258292 26850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:34325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:48:43.268707 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42009
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:43.384303 27146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:43.384559 27146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:43.384603 27146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:43.388679 27146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:43.388748 27146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:43.388813 27146 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:48:43.394218 27146 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27146
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:43.395349 27146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:43.396373 27146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:43.406356 27152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:43.406450 27146 server_base.cc:1034] running on GCE node
W20260806 18:48:43.406513 27158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:43.406387 27154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:43.406962 27146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:43.407657 27146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:43.408815 27146 hybrid_clock.cc:648] HybridClock initialized: now 1786042123408792 us; error 35 us; skew 500 ppm
Aug 06 18:48:43 dist-test-slave-fgb3 krb5kdc[26783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042123, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:43.411603 27146 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:48:43.412533 27146 webserver.cc:460] Webserver started at http://127.1.245.66:34221/ using document root <none> and password file <none>
I20260806 18:48:43.413076 27146 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:43.413134 27146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:43.413300 27146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:43.415472 27146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "c42e046fc0b2497ab902091488b3c902"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "cacc603fd005ac7a6d4e955a2ad5aa7d"
server_key_iv: "9e918a982ba6f7e8e06e94466e6dcbc4"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.416028 27146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "c42e046fc0b2497ab902091488b3c902"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "cacc603fd005ac7a6d4e955a2ad5aa7d"
server_key_iv: "9e918a982ba6f7e8e06e94466e6dcbc4"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.419906 27146 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:48:43.422263 27168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.423492 27146 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:43.423602 27146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "c42e046fc0b2497ab902091488b3c902"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "cacc603fd005ac7a6d4e955a2ad5aa7d"
server_key_iv: "9e918a982ba6f7e8e06e94466e6dcbc4"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.423688 27146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:43.448263 27146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:43.451459 27146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:43.451620 27146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:43.452188 27146 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:43.453012 27146 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:43.453059 27146 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.453099 27146 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:43.453118 27146 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.461474 27146 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:39701
I20260806 18:48:43.461580 27328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:39701 every 8 connection(s)
I20260806 18:48:43.462422 27146 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:48:43.465292 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 27146
I20260806 18:48:43.465431 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:48:43.465720 2005 external_mini_cluster.cc:1428] Setting key e0e64a15fa2f86504764bf7000ff8057
Aug 06 18:48:43 dist-test-slave-fgb3 krb5kdc[26783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042123, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:48:43.476445 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.464241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:46351 (local address 127.1.245.126:42009)
0806 18:48:43.464381 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.464385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.465184 (+ 799us) server_negotiation.cc:407] Connection header received
0806 18:48:43.466573 (+ 1389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.466576 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.466633 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.466705 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:43.467843 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.468364 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.469950 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.470217 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.470334 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.473080 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:43.473084 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:43.473086 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:43.473111 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:43.474464 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:43.474938 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:43.474942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:43.474943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:43.475016 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:43.475330 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:43.475333 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:43.475335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:43.475520 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:43.475608 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.476145 (+ 537us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.476274 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:48:43.477476 27335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.464587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:42009 (local address 127.1.245.66:46351)
0806 18:48:43.465027 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:43.465070 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:43.466374 (+ 1304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:43.466874 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:43.466880 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:43.467223 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:43.467704 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.467717 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.468484 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.468487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.469806 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.469819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.470506 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.470511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.470709 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.471352 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:43.471369 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:43.472927 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:43.474569 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:43.474575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:43.474591 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:43.474838 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:43.475099 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:43.475102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:43.475105 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:43.475241 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:43.475623 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:43.475630 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:43.475909 (+ 279us) client_negotiation.cc:769] Sending connection context
0806 18:48:43.476082 (+ 173us) client_negotiation.cc:240] Negotiation successful
0806 18:48:43.476428 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":118,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
I20260806 18:48:43.478736 27329 heartbeater.cc:347] Connected to a master server at 127.1.245.126:42009
I20260806 18:48:43.479008 27329 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:43.479521 27329 heartbeater.cc:511] Master 127.1.245.126:42009 requested a full tablet report, sending...
I20260806 18:48:43.480682 26850 ts_manager.cc:194] Registered new tserver with Master: c42e046fc0b2497ab902091488b3c902 (127.1.245.66:39701)
I20260806 18:48:43.481244 26850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:46351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:48:43.495496 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42009
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:44063
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:43.609769 27339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:43.610100 27339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:43.610176 27339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:43.614410 27339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:43.614483 27339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:43.614568 27339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:48:43.620335 27339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:44063
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:43.621526 27339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:43.622686 27339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:43.632575 27347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:43.632575 27348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:43.632596 27351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:43.632983 27339 server_base.cc:1034] running on GCE node
I20260806 18:48:43.633416 27339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:43.634037 27339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:43.635198 27339 hybrid_clock.cc:648] HybridClock initialized: now 1786042123635178 us; error 33 us; skew 500 ppm
Aug 06 18:48:43 dist-test-slave-fgb3 krb5kdc[26783](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042123, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:43.637936 27339 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:48:43.638864 27339 webserver.cc:460] Webserver started at http://127.1.245.67:46373/ using document root <none> and password file <none>
I20260806 18:48:43.639421 27339 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:43.639492 27339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:43.639698 27339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:43.641791 27339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "9facc8cc1d874ab298823924fe1fe173"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "ec9e624a5435a690e70159313eb0195f"
server_key_iv: "c8bfd56f0d6ca255a3628704b5f02fd9"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.642414 27339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "9facc8cc1d874ab298823924fe1fe173"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "ec9e624a5435a690e70159313eb0195f"
server_key_iv: "c8bfd56f0d6ca255a3628704b5f02fd9"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.646528 27339 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:48:43.649053 27359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.650457 27339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:48:43.650578 27339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "9facc8cc1d874ab298823924fe1fe173"
format_stamp: "Formatted at 2026-08-06 18:48:43 on dist-test-slave-fgb3"
server_key: "ec9e624a5435a690e70159313eb0195f"
server_key_iv: "c8bfd56f0d6ca255a3628704b5f02fd9"
server_key_version: "encryptionkey@0"
I20260806 18:48:43.650681 27339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:43.662304 27339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:43.665364 27339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:43.665555 27339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:43.666164 27339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:43.667092 27339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:43.667142 27339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.667200 27339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:43.667230 27339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:43.676151 27339 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:35493
I20260806 18:48:43.676179 27535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:35493 every 8 connection(s)
I20260806 18:48:43.677023 27339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:48:43.682466 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 27339
I20260806 18:48:43.682601 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:48:43.682878 2005 external_mini_cluster.cc:1428] Setting key c6b448607e1f8cbacd2b731b149a3375
Aug 06 18:48:43 dist-test-slave-fgb3 krb5kdc[26783](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042123, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:43.690361 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.678805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:42457 (local address 127.1.245.126:42009)
0806 18:48:43.678952 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.678956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.679789 (+ 833us) server_negotiation.cc:407] Connection header received
0806 18:48:43.681110 (+ 1321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.681114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.681171 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.681248 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:43.682482 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.683122 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.684303 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.684524 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.684603 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.687111 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:43.687114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:43.687116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:43.687142 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:43.688333 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:43.688851 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:43.688855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:43.688856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:43.688960 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:43.689328 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:43.689330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:43.689331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:43.689508 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:43.689584 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.690105 (+ 521us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.690209 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:48:43.691257 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.679168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:42009 (local address 127.1.245.67:42457)
0806 18:48:43.679626 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:43.679667 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:43.680892 (+ 1225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:43.681430 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:43.681437 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:43.681788 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:43.682333 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.682354 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.683248 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.683251 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.684197 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.684210 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.684665 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.684669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.684854 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.685347 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:43.685364 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:43.686950 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:43.688456 (+ 1506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:43.688461 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:43.688473 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:43.688738 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:43.689075 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:43.689078 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:43.689079 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:43.689255 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:43.689634 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:43.689639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:43.689902 (+ 263us) client_negotiation.cc:769] Sending connection context
0806 18:48:43.690068 (+ 166us) client_negotiation.cc:240] Negotiation successful
0806 18:48:43.690428 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":116,"threads_started":1}
I20260806 18:48:43.692380 27536 heartbeater.cc:347] Connected to a master server at 127.1.245.126:42009
I20260806 18:48:43.692636 27536 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:43.693156 27536 heartbeater.cc:511] Master 127.1.245.126:42009 requested a full tablet report, sending...
I20260806 18:48:43.694240 26850 ts_manager.cc:194] Registered new tserver with Master: 9facc8cc1d874ab298823924fe1fe173 (127.1.245.67:35493)
I20260806 18:48:43.694857 26850 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:42457
I20260806 18:48:43.696676 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:48:43.705663 26900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.698758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58864 (local address 127.1.245.126:42009)
0806 18:48:43.698892 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.698895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.698951 (+ 56us) server_negotiation.cc:407] Connection header received
0806 18:48:43.699060 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.699062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.699099 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.699161 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:43.699857 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.700381 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.701340 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.701546 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.701618 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.702906 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:43.702909 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:43.702911 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:43.702934 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:43.704181 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:43.704593 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:43.704596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:43.704598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:43.704675 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:43.704891 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:43.704893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:43.704894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:43.705097 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:43.705196 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.705413 (+ 217us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.705506 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260806 18:48:43.708467 26850 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:48:43.711570 26850 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:48:43.725159 27549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.719976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:39701 (local address 127.0.0.1:36864)
0806 18:48:43.720344 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:43.720402 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:43.720549 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:43.721281 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:43.721286 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:43.721309 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:43.721442 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.721452 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.722686 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.722689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.723769 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.723778 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.724612 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.724615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.724733 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.724773 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 18:48:43.724883 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:48:43.724998 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
I20260806 18:48:43.725211 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.720155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36864 (local address 127.1.245.66:39701)
0806 18:48:43.720493 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.720507 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.720532 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:48:43.720709 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.720713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.720904 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.721062 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:43.721578 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.722578 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.723914 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.724489 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.724590 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.724747 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.724936 (+ 189us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.725042 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":115,"threads_started":1}
I20260806 18:48:43.726552 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.721066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53316 (local address 127.1.245.65:45685)
0806 18:48:43.721775 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.721790 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.721823 (+ 33us) server_negotiation.cc:407] Connection header received
0806 18:48:43.721899 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.721904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.722083 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.722237 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:43.722485 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.723636 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.725307 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.725981 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.726080 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.726218 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.726345 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.726428 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":101,"threads_started":1}
I20260806 18:48:43.726588 27551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.720726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:45685 (local address 127.0.0.1:53316)
0806 18:48:43.721438 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:43.721452 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:43.721543 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:43.722228 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:43.722232 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:43.722252 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:43.722376 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.722383 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.723791 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.723794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.725167 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.725179 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.726094 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.726098 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.726194 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.726215 (+ 21us) client_negotiation.cc:769] Sending connection context
0806 18:48:43.726292 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:48:43.726406 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":139,"thread_start_us":91,"threads_started":1}
I20260806 18:48:43.727530 27553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.721079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35493 (local address 127.0.0.1:38056)
0806 18:48:43.721515 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:43.721526 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:43.721591 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:43.722280 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:43.722283 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:43.722334 (+ 51us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:43.722424 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.722430 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.723931 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.723934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.725198 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.725209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.727146 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.727150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.727235 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.727259 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:48:43.727336 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:48:43.727401 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":64,"thread_start_us":93,"threads_started":1}
I20260806 18:48:43.728260 27554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.721524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.1.245.67:35493)
0806 18:48:43.721903 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.721913 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.721931 (+ 18us) server_negotiation.cc:407] Connection header received
0806 18:48:43.721993 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.721998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.722142 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.722579 (+ 437us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:43.722729 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.723801 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.725540 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.727011 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.727846 (+ 835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.728017 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.728057 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.728126 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":180,"threads_started":1}
I20260806 18:48:43.728627 27233 tablet_service.cc:1467] Processing CreateTablet for tablet f712d883ecbf4e049efd8a5ec99d7f57 (DEFAULT_TABLE table=test-table [id=0a8d6b04cff441398b78b7caf0ac969d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:43.729821 27233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f712d883ecbf4e049efd8a5ec99d7f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:43.729558 27032 tablet_service.cc:1467] Processing CreateTablet for tablet f712d883ecbf4e049efd8a5ec99d7f57 (DEFAULT_TABLE table=test-table [id=0a8d6b04cff441398b78b7caf0ac969d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:43.730635 27032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f712d883ecbf4e049efd8a5ec99d7f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:43.730576 27430 tablet_service.cc:1467] Processing CreateTablet for tablet f712d883ecbf4e049efd8a5ec99d7f57 (DEFAULT_TABLE table=test-table [id=0a8d6b04cff441398b78b7caf0ac969d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:43.731359 27430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f712d883ecbf4e049efd8a5ec99d7f57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:43.736838 27557 tablet_bootstrap.cc:492] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Bootstrap starting.
I20260806 18:48:43.738068 27560 tablet_bootstrap.cc:492] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Bootstrap starting.
I20260806 18:48:43.739063 27557 tablet_bootstrap.cc:654] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:43.740089 27557 log.cc:824] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:43.740319 27560 tablet_bootstrap.cc:654] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:43.740968 27560 log.cc:824] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:43.741541 27558 tablet_bootstrap.cc:492] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d: Bootstrap starting.
I20260806 18:48:43.742002 27557 tablet_bootstrap.cc:492] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: No bootstrap required, opened a new log
I20260806 18:48:43.742197 27557 ts_tablet_manager.cc:1397] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:48:43.742908 27560 tablet_bootstrap.cc:492] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: No bootstrap required, opened a new log
I20260806 18:48:43.743095 27560 ts_tablet_manager.cc:1397] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:48:43.743340 27558 tablet_bootstrap.cc:654] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:43.744441 27558 log.cc:824] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:43.746104 27558 tablet_bootstrap.cc:492] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d: No bootstrap required, opened a new log
I20260806 18:48:43.746281 27558 ts_tablet_manager.cc:1397] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:48:43.746099 27557 raft_consensus.cc:348] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.746452 27557 raft_consensus.cc:374] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:43.746517 27557 raft_consensus.cc:729] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9facc8cc1d874ab298823924fe1fe173, State: Initialized, Role: FOLLOWER
I20260806 18:48:43.746768 27560 raft_consensus.cc:348] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.747021 27560 raft_consensus.cc:374] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:43.746987 27557 consensus_queue.cc:260] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.747116 27560 raft_consensus.cc:729] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c42e046fc0b2497ab902091488b3c902, State: Initialized, Role: FOLLOWER
I20260806 18:48:43.747610 27560 consensus_queue.cc:260] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.747696 27536 heartbeater.cc:503] Master 127.1.245.126:42009 was elected leader, sending a full tablet report...
I20260806 18:48:43.748049 27557 ts_tablet_manager.cc:1428] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:48:43.748590 27560 ts_tablet_manager.cc:1428] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:48:43.748628 27329 heartbeater.cc:503] Master 127.1.245.126:42009 was elected leader, sending a full tablet report...
I20260806 18:48:43.749982 27558 raft_consensus.cc:348] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.750231 27558 raft_consensus.cc:374] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:43.750358 27558 raft_consensus.cc:729] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08bc585aa8b642f4b6e4c87ad0117d3d, State: Initialized, Role: FOLLOWER
I20260806 18:48:43.750787 27558 consensus_queue.cc:260] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.751446 27139 heartbeater.cc:503] Master 127.1.245.126:42009 was elected leader, sending a full tablet report...
I20260806 18:48:43.752691 27558 ts_tablet_manager.cc:1428] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260806 18:48:43.928146 27537 tablet.cc:2369] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:43.947547 27568 raft_consensus.cc:482] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:43.947769 27568 raft_consensus.cc:504] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.948993 27568 leader_election.cc:288] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c42e046fc0b2497ab902091488b3c902 (127.1.245.66:39701), 9facc8cc1d874ab298823924fe1fe173 (127.1.245.67:35493)
I20260806 18:48:43.953295 27550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.949346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:52097 (local address 127.1.245.66:39701)
0806 18:48:43.949490 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.949495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.949746 (+ 251us) server_negotiation.cc:407] Connection header received
0806 18:48:43.950024 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.950028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.950077 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.950146 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:43.950557 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.951277 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.952348 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.952761 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.952886 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.952941 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.953098 (+ 157us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.953166 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:48:43.953310 27572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.949241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:39701 (local address 127.1.245.65:52097)
0806 18:48:43.949648 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:43.949663 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:43.949867 (+ 204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:43.950231 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:43.950234 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:43.950253 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:43.950428 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.950435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.951385 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.951388 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.952225 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.952235 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.952876 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.952879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.952970 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.952998 (+ 28us) client_negotiation.cc:769] Sending connection context
0806 18:48:43.953079 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:48:43.953150 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":19,"spinlock_wait_cycles":23168,"thread_start_us":139,"threads_started":1}
I20260806 18:48:43.953998 27554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.949738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:39695 (local address 127.1.245.67:35493)
0806 18:48:43.949891 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:43.949895 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:43.949911 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:48:43.949966 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:43.949969 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:43.950016 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:43.950097 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:43.950440 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.951217 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.952744 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.953285 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.953408 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.953457 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:43.953828 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 18:48:43.953897 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 18:48:43.954015 27569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:43.949291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35493 (local address 127.1.245.65:39695)
0806 18:48:43.949760 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:43.949772 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:43.949866 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:43.950120 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:43.950123 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:43.950142 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:43.950329 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.950339 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.951338 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.951341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.952613 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:43.952624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:43.953525 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:43.953529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:43.953673 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:43.953696 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:48:43.953797 (+ 101us) client_negotiation.cc:240] Negotiation successful
0806 18:48:43.953868 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":65,"spinlock_wait_cycles":6016,"thread_start_us":125,"threads_started":1}
I20260806 18:48:43.954073 27260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f712d883ecbf4e049efd8a5ec99d7f57" candidate_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c42e046fc0b2497ab902091488b3c902" is_pre_election: true
I20260806 18:48:43.954422 27260 raft_consensus.cc:2454] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08bc585aa8b642f4b6e4c87ad0117d3d in term 0.
I20260806 18:48:43.954704 27460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f712d883ecbf4e049efd8a5ec99d7f57" candidate_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9facc8cc1d874ab298823924fe1fe173" is_pre_election: true
I20260806 18:48:43.955006 27460 raft_consensus.cc:2454] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 08bc585aa8b642f4b6e4c87ad0117d3d in term 0.
I20260806 18:48:43.955154 26973 leader_election.cc:302] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bc585aa8b642f4b6e4c87ad0117d3d, c42e046fc0b2497ab902091488b3c902; no voters:
I20260806 18:48:43.955426 27568 raft_consensus.cc:2781] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:48:43.955554 27568 raft_consensus.cc:482] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:48:43.955622 27568 raft_consensus.cc:3037] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:43.956854 27568 raft_consensus.cc:504] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.957204 27568 leader_election.cc:288] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [CANDIDATE]: Term 1 election: Requested vote from peers c42e046fc0b2497ab902091488b3c902 (127.1.245.66:39701), 9facc8cc1d874ab298823924fe1fe173 (127.1.245.67:35493)
I20260806 18:48:43.957535 27460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f712d883ecbf4e049efd8a5ec99d7f57" candidate_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9facc8cc1d874ab298823924fe1fe173"
I20260806 18:48:43.957572 27260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f712d883ecbf4e049efd8a5ec99d7f57" candidate_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c42e046fc0b2497ab902091488b3c902"
I20260806 18:48:43.957645 27460 raft_consensus.cc:3037] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:43.957684 27260 raft_consensus.cc:3037] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:43.958827 27260 raft_consensus.cc:2454] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08bc585aa8b642f4b6e4c87ad0117d3d in term 1.
I20260806 18:48:43.958909 27460 raft_consensus.cc:2454] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 08bc585aa8b642f4b6e4c87ad0117d3d in term 1.
I20260806 18:48:43.959101 26973 leader_election.cc:302] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08bc585aa8b642f4b6e4c87ad0117d3d, c42e046fc0b2497ab902091488b3c902; no voters:
I20260806 18:48:43.959272 27568 raft_consensus.cc:2781] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:43.959506 27568 raft_consensus.cc:686] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [term 1 LEADER]: Becoming Leader. State: Replica: 08bc585aa8b642f4b6e4c87ad0117d3d, State: Running, Role: LEADER
I20260806 18:48:43.959842 27568 consensus_queue.cc:237] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } }
I20260806 18:48:43.962868 26850 catalog_manager.cc:5495] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d reported cstate change: term changed from 0 to 1, leader changed from <none> to 08bc585aa8b642f4b6e4c87ad0117d3d (127.1.245.65). New cstate: current_term: 1 leader_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08bc585aa8b642f4b6e4c87ad0117d3d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 45685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 } health_report { overall_health: UNKNOWN } } }
W20260806 18:48:43.963481 27332 tablet.cc:2369] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:48:43.989634 27140 tablet.cc:2369] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:44.021095 27552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.017499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53322 (local address 127.1.245.65:45685)
0806 18:48:44.017629 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.017633 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.017648 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:48:44.017785 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.017788 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.017861 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.018004 (+ 143us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:48:44.018257 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.018799 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.019688 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.019914 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.019993 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.020245 (+ 252us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:48:44.020673 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:48:44.020762 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.020896 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.020950 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:48:44.027520 27460 raft_consensus.cc:1261] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 1 FOLLOWER]: Refusing update from remote peer 08bc585aa8b642f4b6e4c87ad0117d3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:44.027520 27260 raft_consensus.cc:1261] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 1 FOLLOWER]: Refusing update from remote peer 08bc585aa8b642f4b6e4c87ad0117d3d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:44.028359 27575 consensus_queue.cc:1035] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9facc8cc1d874ab298823924fe1fe173" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:44.028581 27568 consensus_queue.cc:1035] T f712d883ecbf4e049efd8a5ec99d7f57 P 08bc585aa8b642f4b6e4c87ad0117d3d [LEADER]: Connected to new peer: Peer: permanent_uuid: "c42e046fc0b2497ab902091488b3c902" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:44.037825 27583 mvcc.cc:205] Tried to move back new op lower bound from 7315628540008493056 to 7315628539742334976. Current Snapshot: MvccSnapshot[applied={T|T < 7315628540008493056 or (T in {7315628540008493056})}]
I20260806 18:48:44.047806 26847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:44.048020 26847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:44.050872 26847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be077f3b7ba7499eb3e5493380843028: Sending DeleteTablet for 3 replicas of tablet f712d883ecbf4e049efd8a5ec99d7f57
I20260806 18:48:44.051817 27032 tablet_service.cc:1514] Processing DeleteTablet for tablet f712d883ecbf4e049efd8a5ec99d7f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:44 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:53316
I20260806 18:48:44.051816 27233 tablet_service.cc:1514] Processing DeleteTablet for tablet f712d883ecbf4e049efd8a5ec99d7f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:44 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:36864
I20260806 18:48:44.052428 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 26932
I20260806 18:48:44.052392 27430 tablet_service.cc:1514] Processing DeleteTablet for tablet f712d883ecbf4e049efd8a5ec99d7f57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:44 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:38056
I20260806 18:48:44.055120 27597 tablet_replica.cc:330] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: stopping tablet replica
I20260806 18:48:44.055567 27597 raft_consensus.cc:2229] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:44.056031 27597 raft_consensus.cc:2258] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:44.058637 27597 ts_tablet_manager.cc:1905] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:44.060071 27598 tablet_replica.cc:330] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: stopping tablet replica
I20260806 18:48:44.060444 27598 raft_consensus.cc:2229] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:44.060850 27598 raft_consensus.cc:2258] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:44.061297 27597 ts_tablet_manager.cc:1918] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:44.061395 27597 log.cc:1196] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/f712d883ecbf4e049efd8a5ec99d7f57
I20260806 18:48:44.061704 27597 ts_tablet_manager.cc:1939] T f712d883ecbf4e049efd8a5ec99d7f57 P c42e046fc0b2497ab902091488b3c902: Deleting consensus metadata
I20260806 18:48:44.062662 27598 ts_tablet_manager.cc:1905] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:44.062989 26830 catalog_manager.cc:4841] TS c42e046fc0b2497ab902091488b3c902 (127.1.245.66:39701): tablet f712d883ecbf4e049efd8a5ec99d7f57 (table test-table [id=0a8d6b04cff441398b78b7caf0ac969d]) successfully deleted
I20260806 18:48:44.065332 27598 ts_tablet_manager.cc:1918] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:44.065433 27598 log.cc:1196] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/f712d883ecbf4e049efd8a5ec99d7f57
I20260806 18:48:44.065742 27598 ts_tablet_manager.cc:1939] T f712d883ecbf4e049efd8a5ec99d7f57 P 9facc8cc1d874ab298823924fe1fe173: Deleting consensus metadata
I20260806 18:48:44.067157 26831 catalog_manager.cc:4841] TS 9facc8cc1d874ab298823924fe1fe173 (127.1.245.67:35493): tablet f712d883ecbf4e049efd8a5ec99d7f57 (table test-table [id=0a8d6b04cff441398b78b7caf0ac969d]) successfully deleted
I20260806 18:48:44.067595 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 27146
W20260806 18:48:44.067580 26830 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:45685 (error 108)
I20260806 18:48:44.078830 27553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.078510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:45685 (local address 127.0.0.1:53324)
0806 18:48:44.078637 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:44.078713 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:45685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 18:48:44.079041 26830 catalog_manager.cc:4584] TS 08bc585aa8b642f4b6e4c87ad0117d3d (127.1.245.65:45685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f712d883ecbf4e049efd8a5ec99d7f57: Network error: Client connection negotiation failed: client connection to 127.1.245.65:45685: connect: Connection refused (error 111)
I20260806 18:48:44.083390 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 27339
I20260806 18:48:44.096910 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 26800
2026-08-06T18:48:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2589 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T18:48:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:44Z Disabled control of system clock
I20260806 18:48:44.132524 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:39037
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:35347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:39037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:48:44.247224 27610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:44.247488 27610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:44.247525 27610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:44.251485 27610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:44.251539 27610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:44.251554 27610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:44.251569 27610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:44.251581 27610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:44.256291 27610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:35347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:39037
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:39037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:44.257344 27610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:44.258811 27610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:44.267127 27618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:44.267159 27620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:44.267127 27616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:44.267455 27610 server_base.cc:1034] running on GCE node
I20260806 18:48:44.268110 27610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:44.269182 27610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:44.270362 27610 hybrid_clock.cc:648] HybridClock initialized: now 1786042124270344 us; error 32 us; skew 500 ppm
I20260806 18:48:44.272212 27610 webserver.cc:460] Webserver started at http://127.1.245.126:41309/ using document root <none> and password file <none>
I20260806 18:48:44.272800 27610 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:44.272862 27610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:44.273061 27610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:44.275178 27610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "3189e232ecd646a38d36a698f1bf59d9"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "968b58ece56678af0553e45ba5b69781"
server_key_iv: "e46690bfa5bdeebde95e2ace726c2c2d"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.275822 27610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "3189e232ecd646a38d36a698f1bf59d9"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "968b58ece56678af0553e45ba5b69781"
server_key_iv: "e46690bfa5bdeebde95e2ace726c2c2d"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.279495 27610 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:48:44.282004 27629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.283252 27610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:44.283370 27610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "3189e232ecd646a38d36a698f1bf59d9"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "968b58ece56678af0553e45ba5b69781"
server_key_iv: "e46690bfa5bdeebde95e2ace726c2c2d"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.283475 27610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:44.304765 27610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:44.305577 27610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:44.305773 27610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:44.312935 27610 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:39037
I20260806 18:48:44.312950 27701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:39037 every 8 connection(s)
I20260806 18:48:44.313920 27610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:44.317097 27703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:44.318607 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 27610
I20260806 18:48:44.318742 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:44.319062 2005 external_mini_cluster.cc:1428] Setting key bca172c6cf4c52852f79ce718f9cbdab
I20260806 18:48:44.323731 27703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9: Bootstrap starting.
I20260806 18:48:44.326032 27703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:44.326880 27703 log.cc:824] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:44.328167 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.320528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.1.245.126:39037)
0806 18:48:44.321226 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.321246 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.321283 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:48:44.321932 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.321948 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.322405 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.322719 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:44.323150 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.324138 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.325017 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.325486 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.325630 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.326495 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:44.326500 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:44.326512 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:44.326883 (+ 371us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:44.326945 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:44.326968 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:44.327097 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.327232 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.327484 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"spinlock_wait_cycles":25856,"thread_start_us":219,"threads_started":1}
I20260806 18:48:44.328935 27703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9: No bootstrap required, opened a new log
I20260806 18:48:44.332409 27703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3189e232ecd646a38d36a698f1bf59d9" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39037 } }
I20260806 18:48:44.332633 27703 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:44.332690 27703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3189e232ecd646a38d36a698f1bf59d9, State: Initialized, Role: FOLLOWER
I20260806 18:48:44.333091 27703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3189e232ecd646a38d36a698f1bf59d9" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39037 } }
I20260806 18:48:44.333236 27703 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:44.333307 27703 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:44.333412 27703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:44.334527 27703 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3189e232ecd646a38d36a698f1bf59d9" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39037 } }
I20260806 18:48:44.334904 27703 leader_election.cc:302] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3189e232ecd646a38d36a698f1bf59d9; no voters:
I20260806 18:48:44.335246 27703 leader_election.cc:288] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:44.335420 27712 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:44.335652 27712 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [term 1 LEADER]: Becoming Leader. State: Replica: 3189e232ecd646a38d36a698f1bf59d9, State: Running, Role: LEADER
I20260806 18:48:44.335970 27712 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3189e232ecd646a38d36a698f1bf59d9" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39037 } }
I20260806 18:48:44.336515 27703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:44.337769 27714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3189e232ecd646a38d36a698f1bf59d9. Latest consensus state: current_term: 1 leader_uuid: "3189e232ecd646a38d36a698f1bf59d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3189e232ecd646a38d36a698f1bf59d9" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39037 } } }
I20260806 18:48:44.337910 27714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:44.337780 27713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3189e232ecd646a38d36a698f1bf59d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3189e232ecd646a38d36a698f1bf59d9" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39037 } } }
I20260806 18:48:44.338040 27713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:44.338352 27724 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:44.341133 27724 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:44.346733 27724 catalog_manager.cc:1347] Generated new cluster ID: 49148f7c419c43448aea3fc771b5f053
I20260806 18:48:44.346800 27724 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:44.361224 27724 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:44.362205 27724 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:44.366194 27724 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9: Generated new TSK 0
I20260806 18:48:44.366884 27724 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:48:44.371404 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39037
--builtin_ntp_servers=127.1.245.84:35347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:48:44.485481 27739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:44.485726 27739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:44.485762 27739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:44.489864 27739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:44.489930 27739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:44.489984 27739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:48:44.494794 27739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:35347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:44.495893 27739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:44.497334 27739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:44.507634 27750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:44.507634 27749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:44.507624 27757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:44.507817 27739 server_base.cc:1034] running on GCE node
I20260806 18:48:44.508270 27739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:44.508884 27739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:44.510047 27739 hybrid_clock.cc:648] HybridClock initialized: now 1786042124510029 us; error 27 us; skew 500 ppm
I20260806 18:48:44.511911 27739 webserver.cc:460] Webserver started at http://127.1.245.65:34983/ using document root <none> and password file <none>
I20260806 18:48:44.512470 27739 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:44.512532 27739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:44.512734 27739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:44.514789 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "4167f5d345af4148bdd6dde30a7c0914"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "2c72ef380bef5738bfb1895f72d7637d"
server_key_iv: "cc151b386b07d1d3f0aa821af83d1fc4"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.515316 27739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "4167f5d345af4148bdd6dde30a7c0914"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "2c72ef380bef5738bfb1895f72d7637d"
server_key_iv: "cc151b386b07d1d3f0aa821af83d1fc4"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.518939 27739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:48:44.521292 27764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.522601 27739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:44.522723 27739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "4167f5d345af4148bdd6dde30a7c0914"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "2c72ef380bef5738bfb1895f72d7637d"
server_key_iv: "cc151b386b07d1d3f0aa821af83d1fc4"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.522820 27739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:44.534947 27739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:44.535640 27739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:44.535811 27739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:44.536449 27739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:44.537442 27739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:44.537492 27739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.537546 27739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:44.537572 27739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.546341 27739 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:41127
I20260806 18:48:44.546383 27932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:41127 every 8 connection(s)
I20260806 18:48:44.547266 27739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:48:44.548318 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 27739
I20260806 18:48:44.548435 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:48:44.548731 2005 external_mini_cluster.cc:1428] Setting key 0658c51221c57d12959ba37558fd4957
I20260806 18:48:44.551016 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39037
--builtin_ntp_servers=127.1.245.84:35347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:48:44.557569 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.549204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:47119 (local address 127.1.245.126:39037)
0806 18:48:44.549369 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.549374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.550692 (+ 1318us) server_negotiation.cc:407] Connection header received
0806 18:48:44.552437 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.552440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.552489 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.552582 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:44.553489 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.554144 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.555372 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.555710 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.555821 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.557018 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:44.557023 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:44.557026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:44.557043 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:44.557068 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:44.557079 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:44.557159 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.557330 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.557410 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 18:48:44.558554 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.549691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:39037 (local address 127.1.245.65:47119)
0806 18:48:44.550530 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:44.550587 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:44.552222 (+ 1635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:44.552765 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:44.552771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:44.552863 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:44.553353 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.553365 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.554252 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.554254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.555236 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.555247 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.555887 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.555890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.556036 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.556724 (+ 688us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:44.556725 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:44.556778 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:44.556853 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:44.557169 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:44.557177 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:44.557182 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:48:44.557313 (+ 131us) client_negotiation.cc:240] Negotiation successful
0806 18:48:44.557605 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":140,"threads_started":1}
I20260806 18:48:44.559846 27933 heartbeater.cc:347] Connected to a master server at 127.1.245.126:39037
I20260806 18:48:44.560070 27933 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:44.560587 27933 heartbeater.cc:511] Master 127.1.245.126:39037 requested a full tablet report, sending...
I20260806 18:48:44.562175 27657 ts_manager.cc:194] Registered new tserver with Master: 4167f5d345af4148bdd6dde30a7c0914 (127.1.245.65:41127)
I20260806 18:48:44.563397 27657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.65:47119
W20260806 18:48:44.664280 27938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:44.664554 27938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:44.664608 27938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:44.668648 27938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:44.668704 27938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:44.668779 27938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:48:44.673578 27938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:35347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:44.674682 27938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:44.676105 27938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:44.685753 27947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:44.685822 27944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:44.685782 27945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:44.685926 27938 server_base.cc:1034] running on GCE node
I20260806 18:48:44.686440 27938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:44.687109 27938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:44.688266 27938 hybrid_clock.cc:648] HybridClock initialized: now 1786042124688250 us; error 29 us; skew 500 ppm
I20260806 18:48:44.690014 27938 webserver.cc:460] Webserver started at http://127.1.245.66:40581/ using document root <none> and password file <none>
I20260806 18:48:44.690745 27938 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:44.690837 27938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:44.691047 27938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:44.693071 27938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "e67b0458dca1471489592f2fb9dee3af"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "95bceead46c6b768b6a4331028046509"
server_key_iv: "73e7047f95b5c9181a298786c44c294a"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.693617 27938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "e67b0458dca1471489592f2fb9dee3af"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "95bceead46c6b768b6a4331028046509"
server_key_iv: "73e7047f95b5c9181a298786c44c294a"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.697377 27938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:48:44.699841 27955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.701020 27938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:48:44.701128 27938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "e67b0458dca1471489592f2fb9dee3af"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "95bceead46c6b768b6a4331028046509"
server_key_iv: "73e7047f95b5c9181a298786c44c294a"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.701222 27938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:44.717229 27938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:44.717897 27938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:44.718086 27938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:44.718690 27938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:44.719674 27938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:44.719724 27938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.719779 27938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:44.719805 27938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.728228 27938 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:36411
I20260806 18:48:44.728250 28122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:36411 every 8 connection(s)
I20260806 18:48:44.729135 27938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:48:44.738103 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.730950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:60103 (local address 127.1.245.126:39037)
0806 18:48:44.731123 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.731128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.732175 (+ 1047us) server_negotiation.cc:407] Connection header received
0806 18:48:44.733494 (+ 1319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.733497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.733545 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.733616 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:44.734494 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.735006 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.736200 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.736443 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.736523 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.737626 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:44.737629 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:44.737631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:44.737644 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:44.737664 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:44.737672 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:44.737728 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.737918 (+ 190us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.737986 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 18:48:44.738667 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 27938
I20260806 18:48:44.738766 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:48:44.739029 2005 external_mini_cluster.cc:1428] Setting key bf96c4876cec9d429c8e193a022e4f23
I20260806 18:48:44.739037 28126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.731390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:39037 (local address 127.1.245.66:60103)
0806 18:48:44.732033 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:44.732074 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:44.733315 (+ 1241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:44.733769 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:44.733775 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:44.733857 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:44.734382 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.734398 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.735136 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.735139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.736069 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.736080 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.736592 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.736595 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.736809 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.737377 (+ 568us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:44.737379 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:44.737420 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:44.737485 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:44.737763 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:44.737773 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:44.737780 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:48:44.737898 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:48:44.738146 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
I20260806 18:48:44.740128 28123 heartbeater.cc:347] Connected to a master server at 127.1.245.126:39037
I20260806 18:48:44.740392 28123 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:44.740981 28123 heartbeater.cc:511] Master 127.1.245.126:39037 requested a full tablet report, sending...
I20260806 18:48:44.741161 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39037
--builtin_ntp_servers=127.1.245.84:35347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:48:44.742040 27656 ts_manager.cc:194] Registered new tserver with Master: e67b0458dca1471489592f2fb9dee3af (127.1.245.66:36411)
I20260806 18:48:44.742731 27656 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.66:60103
W20260806 18:48:44.853623 28127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:44.853884 28127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:44.853921 28127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:44.857867 28127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:44.857920 28127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:44.857993 28127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:48:44.862725 28127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:35347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:39037
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:44.863718 28127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:44.865075 28127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:44.874843 28136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:44.874976 28138 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:44.875056 28127 server_base.cc:1034] running on GCE node
W20260806 18:48:44.875277 28141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:44.875684 28127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:44.876349 28127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:44.877539 28127 hybrid_clock.cc:648] HybridClock initialized: now 1786042124877517 us; error 29 us; skew 500 ppm
I20260806 18:48:44.879503 28127 webserver.cc:460] Webserver started at http://127.1.245.67:34863/ using document root <none> and password file <none>
I20260806 18:48:44.880074 28127 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:44.880148 28127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:44.880374 28127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:44.882676 28127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "71573b023ea843b3b3332c4c31343aac"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "72bdbca0832d4a684738648c975dc278"
server_key_iv: "63237cf849615564abab50d21fd52a11"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.883272 28127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "71573b023ea843b3b3332c4c31343aac"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "72bdbca0832d4a684738648c975dc278"
server_key_iv: "63237cf849615564abab50d21fd52a11"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.887105 28127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:48:44.889775 28150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.891132 28127 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:48:44.891296 28127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "71573b023ea843b3b3332c4c31343aac"
format_stamp: "Formatted at 2026-08-06 18:48:44 on dist-test-slave-fgb3"
server_key: "72bdbca0832d4a684738648c975dc278"
server_key_iv: "63237cf849615564abab50d21fd52a11"
server_key_version: "encryptionkey@0"
I20260806 18:48:44.891412 28127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:44.926054 28127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:44.926869 28127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:44.927098 28127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:44.927824 28127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:44.928853 28127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:44.928910 28127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.928972 28127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:44.929003 28127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:44.938548 28127 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:43513
I20260806 18:48:44.938575 28318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:43513 every 8 connection(s)
I20260806 18:48:44.939548 28127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:48:44.948231 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 28127
I20260806 18:48:44.948369 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:48:44.948680 2005 external_mini_cluster.cc:1428] Setting key 5897968aa90760426d124ea6bd77e852
I20260806 18:48:44.948933 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.941476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:37839 (local address 127.1.245.126:39037)
0806 18:48:44.941650 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.941655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.942730 (+ 1075us) server_negotiation.cc:407] Connection header received
0806 18:48:44.944115 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.944118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.944168 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.944243 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:44.945126 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.945690 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.946863 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.947085 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.947168 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.948427 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:44.948431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:44.948434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:44.948451 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:44.948475 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:44.948486 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:44.948566 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.948709 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.948780 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 18:48:44.949947 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.941848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:39037 (local address 127.1.245.67:37839)
0806 18:48:44.942578 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:44.942621 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:44.943892 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:44.944391 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:44.944397 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:44.944495 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:44.944991 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.945003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.945791 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.945794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.946753 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.946763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.947298 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.947302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.947470 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.948089 (+ 619us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:44.948091 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:44.948132 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:44.948210 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:44.948569 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:44.948577 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:44.948582 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:48:44.948684 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 18:48:44.948946 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":116,"threads_started":1}
I20260806 18:48:44.951187 28319 heartbeater.cc:347] Connected to a master server at 127.1.245.126:39037
I20260806 18:48:44.951438 28319 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:44.951975 28319 heartbeater.cc:511] Master 127.1.245.126:39037 requested a full tablet report, sending...
I20260806 18:48:44.953020 27657 ts_manager.cc:194] Registered new tserver with Master: 71573b023ea843b3b3332c4c31343aac (127.1.245.67:43513)
I20260806 18:48:44.953599 27657 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.67:37839
I20260806 18:48:44.963318 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:48:44.969300 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.965490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41468 (local address 127.1.245.126:39037)
0806 18:48:44.965612 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.965616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.965749 (+ 133us) server_negotiation.cc:407] Connection header received
0806 18:48:44.965864 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.965866 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.965909 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.965988 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:44.966303 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.966864 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.967752 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.967978 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.968067 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.968937 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:44.968942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:44.968944 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:44.968956 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:44.968977 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:44.968985 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:44.969043 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.969129 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.969184 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:48:44.972177 27657 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:48:44.975183 27657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:48:44.989476 28345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.984214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:36411 (local address 127.0.0.1:45828)
0806 18:48:44.984906 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:44.984919 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:44.984997 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:44.985597 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:44.985599 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:44.985615 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:44.985771 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.985776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.986812 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.986815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.988013 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.988023 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.988915 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.988919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.989025 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.989067 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 18:48:44.989182 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:48:44.989304 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":636,"mutex_wait_us":32,"thread_start_us":89,"threads_started":1}
I20260806 18:48:44.989488 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.984366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45828 (local address 127.1.245.66:36411)
0806 18:48:44.984776 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.984791 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.985067 (+ 276us) server_negotiation.cc:407] Connection header received
0806 18:48:44.985136 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.985140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.985310 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.985466 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:44.985892 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.986716 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.988137 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.988793 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.988908 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.989108 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.989214 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.989321 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":105,"threads_started":1}
I20260806 18:48:44.990137 28336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.983505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:41127 (local address 127.0.0.1:35264)
0806 18:48:44.984621 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:44.984674 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:44.984808 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:44.985564 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:44.985571 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:44.985605 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:44.985804 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.985811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.987258 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.987261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.988433 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.988444 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.989655 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.989658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.989831 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.989855 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:48:44.989930 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:48:44.990001 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":124,"spinlock_wait_cycles":6272,"thread_start_us":74,"threads_started":1}
I20260806 18:48:44.990235 28337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.983672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35264 (local address 127.1.245.65:41127)
0806 18:48:44.984575 (+ 903us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.984588 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.984979 (+ 391us) server_negotiation.cc:407] Connection header received
0806 18:48:44.985051 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.985055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.985251 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.985381 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:44.985939 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.987137 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.988554 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.989292 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.989401 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.989590 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.989964 (+ 374us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.990063 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":790,"thread_start_us":144,"threads_started":1}
I20260806 18:48:44.990904 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.983752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:43513 (local address 127.0.0.1:52786)
0806 18:48:44.984721 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:44.984735 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:44.984830 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:44.985564 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:44.985570 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:44.985607 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:44.985804 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.985811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.987425 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.987428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.988537 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:44.988545 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.989664 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.989668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:44.990599 (+ 931us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.990623 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:48:44.990694 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:48:44.990765 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":819,"spinlock_wait_cycles":5504,"thread_start_us":107,"threads_started":1}
I20260806 18:48:44.991099 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:44.983951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52786 (local address 127.1.245.67:43513)
0806 18:48:44.984898 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:44.984911 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:44.984935 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:48:44.985016 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:44.985022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:44.985227 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:44.985382 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:44.985939 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.987305 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.988642 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:44.989325 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:44.989420 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:44.989596 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:44.990845 (+ 1249us) server_negotiation.cc:299] Negotiation successful
0806 18:48:44.990944 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":836,"thread_start_us":142,"threads_started":1}
I20260806 18:48:44.992784 28026 tablet_service.cc:1467] Processing CreateTablet for tablet 0f0fe6044623486480079bb5ba29aabd (DEFAULT_TABLE table=test-table [id=0671441470ba47f9927ee8f09c6c4a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:44.993327 27840 tablet_service.cc:1467] Processing CreateTablet for tablet 0f0fe6044623486480079bb5ba29aabd (DEFAULT_TABLE table=test-table [id=0671441470ba47f9927ee8f09c6c4a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:44.993829 28026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f0fe6044623486480079bb5ba29aabd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:44.994357 27840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f0fe6044623486480079bb5ba29aabd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:44.996004 28222 tablet_service.cc:1467] Processing CreateTablet for tablet 0f0fe6044623486480079bb5ba29aabd (DEFAULT_TABLE table=test-table [id=0671441470ba47f9927ee8f09c6c4a0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:44.997124 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f0fe6044623486480079bb5ba29aabd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:45.000542 28347 tablet_bootstrap.cc:492] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: Bootstrap starting.
I20260806 18:48:45.002847 28348 tablet_bootstrap.cc:492] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914: Bootstrap starting.
I20260806 18:48:45.002882 28347 tablet_bootstrap.cc:654] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:45.003785 28347 log.cc:824] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:45.004649 28350 tablet_bootstrap.cc:492] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: Bootstrap starting.
I20260806 18:48:45.005565 28348 tablet_bootstrap.cc:654] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:45.006202 28347 tablet_bootstrap.cc:492] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: No bootstrap required, opened a new log
I20260806 18:48:45.006307 28348 log.cc:824] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:45.006455 28347 ts_tablet_manager.cc:1397] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260806 18:48:45.007125 28350 tablet_bootstrap.cc:654] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:45.007982 28350 log.cc:824] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:45.008219 28348 tablet_bootstrap.cc:492] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914: No bootstrap required, opened a new log
I20260806 18:48:45.008426 28348 ts_tablet_manager.cc:1397] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:48:45.010015 28350 tablet_bootstrap.cc:492] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: No bootstrap required, opened a new log
I20260806 18:48:45.010212 28350 ts_tablet_manager.cc:1397] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:48:45.011524 28347 raft_consensus.cc:348] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.011807 28347 raft_consensus.cc:374] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:45.011906 28347 raft_consensus.cc:729] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e67b0458dca1471489592f2fb9dee3af, State: Initialized, Role: FOLLOWER
I20260806 18:48:45.011835 28348 raft_consensus.cc:348] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.012084 28348 raft_consensus.cc:374] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:45.012149 28348 raft_consensus.cc:729] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4167f5d345af4148bdd6dde30a7c0914, State: Initialized, Role: FOLLOWER
I20260806 18:48:45.012434 28347 consensus_queue.cc:260] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.012566 28348 consensus_queue.cc:260] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.013144 28123 heartbeater.cc:503] Master 127.1.245.126:39037 was elected leader, sending a full tablet report...
I20260806 18:48:45.013499 28347 ts_tablet_manager.cc:1428] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:48:45.014226 28350 raft_consensus.cc:348] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.014572 28350 raft_consensus.cc:374] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:45.014638 28350 raft_consensus.cc:729] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71573b023ea843b3b3332c4c31343aac, State: Initialized, Role: FOLLOWER
I20260806 18:48:45.014703 27933 heartbeater.cc:503] Master 127.1.245.126:39037 was elected leader, sending a full tablet report...
I20260806 18:48:45.015074 28350 consensus_queue.cc:260] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.015789 28348 ts_tablet_manager.cc:1428] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260806 18:48:45.016038 28350 ts_tablet_manager.cc:1428] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260806 18:48:45.016429 28319 heartbeater.cc:503] Master 127.1.245.126:39037 was elected leader, sending a full tablet report...
W20260806 18:48:45.048559 27935 tablet.cc:2369] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:45.055460 28361 raft_consensus.cc:482] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:45.055678 28361 raft_consensus.cc:504] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.056910 28361 leader_election.cc:288] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4167f5d345af4148bdd6dde30a7c0914 (127.1.245.65:41127), e67b0458dca1471489592f2fb9dee3af (127.1.245.66:36411)
I20260806 18:48:45.061005 28337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:45.057326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:58307 (local address 127.1.245.65:41127)
0806 18:48:45.057463 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:45.057469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:45.057484 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:48:45.057540 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:45.057542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:45.057584 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:45.057649 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:45.058088 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.058755 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.059974 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.060456 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.060548 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:45.060588 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:45.060840 (+ 252us) server_negotiation.cc:299] Negotiation successful
0806 18:48:45.060893 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:48:45.061033 28323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:45.057150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:41127 (local address 127.1.245.67:58307)
0806 18:48:45.057294 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:45.057308 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:45.057422 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:45.057778 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:45.057781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:45.057800 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:45.057938 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:45.057960 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.058881 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.058884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:45.059843 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:45.059853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.060605 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.060609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:45.060711 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:45.060738 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 18:48:45.060825 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 18:48:45.060895 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 18:48:45.061525 28362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:45.057442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:36411 (local address 127.1.245.67:51175)
0806 18:48:45.057809 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:45.057822 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:45.057913 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:45.058152 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:45.058155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:45.058171 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:45.058299 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:45.058369 (+ 70us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.059298 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.059307 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:45.060131 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:45.060142 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.061078 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.061082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:45.061173 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:45.061199 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:48:45.061322 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 18:48:45.061391 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":157,"threads_started":1}
I20260806 18:48:45.061527 28346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:45.057573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:51175 (local address 127.1.245.66:36411)
0806 18:48:45.057725 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:45.057730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:45.057899 (+ 169us) server_negotiation.cc:407] Connection header received
0806 18:48:45.058004 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:45.058007 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:45.058063 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:45.058120 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:45.058495 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.059178 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.060294 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.060926 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.061036 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:45.061084 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:45.061348 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 18:48:45.061423 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260806 18:48:45.061825 27865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0fe6044623486480079bb5ba29aabd" candidate_uuid: "71573b023ea843b3b3332c4c31343aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4167f5d345af4148bdd6dde30a7c0914" is_pre_election: true
I20260806 18:48:45.062140 27865 raft_consensus.cc:2454] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71573b023ea843b3b3332c4c31343aac in term 0.
I20260806 18:48:45.062192 28052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0fe6044623486480079bb5ba29aabd" candidate_uuid: "71573b023ea843b3b3332c4c31343aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e67b0458dca1471489592f2fb9dee3af" is_pre_election: true
I20260806 18:48:45.062497 28052 raft_consensus.cc:2454] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 71573b023ea843b3b3332c4c31343aac in term 0.
I20260806 18:48:45.062657 28151 leader_election.cc:302] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4167f5d345af4148bdd6dde30a7c0914, 71573b023ea843b3b3332c4c31343aac; no voters:
I20260806 18:48:45.062935 28361 raft_consensus.cc:2781] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:48:45.063011 28361 raft_consensus.cc:482] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:48:45.063041 28361 raft_consensus.cc:3037] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:45.064076 28361 raft_consensus.cc:504] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.064421 28361 leader_election.cc:288] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [CANDIDATE]: Term 1 election: Requested vote from peers 4167f5d345af4148bdd6dde30a7c0914 (127.1.245.65:41127), e67b0458dca1471489592f2fb9dee3af (127.1.245.66:36411)
I20260806 18:48:45.064770 27865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0fe6044623486480079bb5ba29aabd" candidate_uuid: "71573b023ea843b3b3332c4c31343aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4167f5d345af4148bdd6dde30a7c0914"
I20260806 18:48:45.064859 28052 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f0fe6044623486480079bb5ba29aabd" candidate_uuid: "71573b023ea843b3b3332c4c31343aac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e67b0458dca1471489592f2fb9dee3af"
I20260806 18:48:45.064903 27865 raft_consensus.cc:3037] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:45.064990 28052 raft_consensus.cc:3037] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:45.066150 27865 raft_consensus.cc:2454] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71573b023ea843b3b3332c4c31343aac in term 1.
I20260806 18:48:45.066150 28052 raft_consensus.cc:2454] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 71573b023ea843b3b3332c4c31343aac in term 1.
I20260806 18:48:45.066475 28151 leader_election.cc:302] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71573b023ea843b3b3332c4c31343aac, e67b0458dca1471489592f2fb9dee3af; no voters:
I20260806 18:48:45.066712 28361 raft_consensus.cc:2781] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:45.066946 28361 raft_consensus.cc:686] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 1 LEADER]: Becoming Leader. State: Replica: 71573b023ea843b3b3332c4c31343aac, State: Running, Role: LEADER
I20260806 18:48:45.067534 28361 consensus_queue.cc:237] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } }
I20260806 18:48:45.070595 27656 catalog_manager.cc:5495] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac reported cstate change: term changed from 0 to 1, leader changed from <none> to 71573b023ea843b3b3332c4c31343aac (127.1.245.67). New cstate: current_term: 1 leader_uuid: "71573b023ea843b3b3332c4c31343aac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "71573b023ea843b3b3332c4c31343aac" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 43513 } health_report { overall_health: HEALTHY } } }
I20260806 18:48:45.093006 28344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:45.089435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52790 (local address 127.1.245.67:43513)
0806 18:48:45.089576 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:45.089581 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:45.089596 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:48:45.089691 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:45.089694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:45.089752 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:45.089852 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:48:45.090096 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.090668 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.091516 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:45.091729 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:45.091819 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:45.092125 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:48:45.092603 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:48:45.092727 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:45.092796 (+ 69us) server_negotiation.cc:299] Negotiation successful
0806 18:48:45.092855 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260806 18:48:45.099087 28052 raft_consensus.cc:1261] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 1 FOLLOWER]: Refusing update from remote peer 71573b023ea843b3b3332c4c31343aac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:45.099133 27865 raft_consensus.cc:1261] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914 [term 1 FOLLOWER]: Refusing update from remote peer 71573b023ea843b3b3332c4c31343aac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:45.099817 28361 consensus_queue.cc:1035] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [LEADER]: Connected to new peer: Peer: permanent_uuid: "e67b0458dca1471489592f2fb9dee3af" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 36411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:45.100018 28364 consensus_queue.cc:1035] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [LEADER]: Connected to new peer: Peer: permanent_uuid: "4167f5d345af4148bdd6dde30a7c0914" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 41127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:45.107884 28371 mvcc.cc:205] Tried to move back new op lower bound from 7315628544398610432 to 7315628544279326720. Current Snapshot: MvccSnapshot[applied={T|T < 7315628544398610432}]
I20260806 18:48:45.108566 28368 mvcc.cc:205] Tried to move back new op lower bound from 7315628544398610432 to 7315628544279326720. Current Snapshot: MvccSnapshot[applied={T|T < 7315628544398610432}]
I20260806 18:48:45.115831 27657 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:45.116007 27657 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:45.118404 27657 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3189e232ecd646a38d36a698f1bf59d9: Sending DeleteTablet for 3 replicas of tablet 0f0fe6044623486480079bb5ba29aabd
I20260806 18:48:45.119199 28222 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f0fe6044623486480079bb5ba29aabd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:45 UTC) from {username='slave'} at 127.0.0.1:52786
I20260806 18:48:45.119347 27840 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f0fe6044623486480079bb5ba29aabd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:45 UTC) from {username='slave'} at 127.0.0.1:35264
I20260806 18:48:45.119519 28026 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f0fe6044623486480079bb5ba29aabd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:45 UTC) from {username='slave'} at 127.0.0.1:45828
I20260806 18:48:45.119752 28384 tablet_replica.cc:330] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: stopping tablet replica
I20260806 18:48:45.120003 28387 tablet_replica.cc:330] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: stopping tablet replica
I20260806 18:48:45.120239 28385 tablet_replica.cc:330] T 0f0fe6044623486480079bb5ba29aabd P 4167f5d345af4148bdd6dde30a7c0914: stopping tablet replica
I20260806 18:48:45.120385 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 27739
I20260806 18:48:45.120404 28387 raft_consensus.cc:2229] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:45.120675 28387 raft_consensus.cc:2258] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:45.122893 28384 raft_consensus.cc:2229] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:48:45.123234 28384 raft_consensus.cc:2258] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:45.124431 28387 ts_tablet_manager.cc:1905] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:45.127723 28387 ts_tablet_manager.cc:1918] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:45.127918 28387 log.cc:1196] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/0f0fe6044623486480079bb5ba29aabd
I20260806 18:48:45.128386 28387 ts_tablet_manager.cc:1939] T 0f0fe6044623486480079bb5ba29aabd P e67b0458dca1471489592f2fb9dee3af: Deleting consensus metadata
I20260806 18:48:45.128953 28384 ts_tablet_manager.cc:1905] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:45.129794 27630 catalog_manager.cc:4841] TS e67b0458dca1471489592f2fb9dee3af (127.1.245.66:36411): tablet 0f0fe6044623486480079bb5ba29aabd (table test-table [id=0671441470ba47f9927ee8f09c6c4a0f]) successfully deleted
I20260806 18:48:45.131661 28384 ts_tablet_manager.cc:1918] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:45.131765 28384 log.cc:1196] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/0f0fe6044623486480079bb5ba29aabd
I20260806 18:48:45.132112 28384 ts_tablet_manager.cc:1939] T 0f0fe6044623486480079bb5ba29aabd P 71573b023ea843b3b3332c4c31343aac: Deleting consensus metadata
I20260806 18:48:45.133327 27634 catalog_manager.cc:4841] TS 71573b023ea843b3b3332c4c31343aac (127.1.245.67:43513): tablet 0f0fe6044623486480079bb5ba29aabd (table test-table [id=0671441470ba47f9927ee8f09c6c4a0f]) successfully deleted
W20260806 18:48:45.134797 27630 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:41127 (error 108)
I20260806 18:48:45.134932 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 27938
I20260806 18:48:45.138552 28341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:45.135617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:41127 (local address 127.0.0.1:35266)
0806 18:48:45.138367 (+ 2750us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:45.138455 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:41127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2622}
W20260806 18:48:45.138731 27630 catalog_manager.cc:4584] TS 4167f5d345af4148bdd6dde30a7c0914 (127.1.245.65:41127): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f0fe6044623486480079bb5ba29aabd: Network error: Client connection negotiation failed: client connection to 127.1.245.65:41127: connect: Connection refused (error 111)
I20260806 18:48:45.148277 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 28127
I20260806 18:48:45.161171 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 27610
2026-08-06T18:48:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1065 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:45 dist-test-slave-fgb3 krb5kdc[28391](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:45 dist-test-slave-fgb3 krb5kdc[28391](info): set up 2 sockets
Aug 06 18:48:45 dist-test-slave-fgb3 krb5kdc[28391](info): commencing operation
krb5kdc: starting...
W20260806 18:48:46.289594 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:48:46 dist-test-slave-fgb3 krb5kdc[28391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:48:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:48:46.414042 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:38337
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:38337
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:46.528579 28411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:46.528872 28411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:46.528924 28411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:46.532929 28411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:46.532994 28411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:46.533012 28411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:46.533028 28411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:46.533042 28411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:46.538594 28411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:38337
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:38337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:46.539734 28411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:46.540779 28411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:46.550278 28422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:46.550866 28423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:46.551517 28426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:46.551573 28411 server_base.cc:1034] running on GCE node
I20260806 18:48:46.552249 28411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:46.553514 28411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:46.554713 28411 hybrid_clock.cc:648] HybridClock initialized: now 1786042126554709 us; error 42 us; skew 500 ppm
Aug 06 18:48:46 dist-test-slave-fgb3 krb5kdc[28391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042126, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:46.557716 28411 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:48:46.558774 28411 webserver.cc:460] Webserver started at http://127.1.245.126:36377/ using document root <none> and password file <none>
I20260806 18:48:46.559351 28411 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:46.559422 28411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:46.559643 28411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:46.561848 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "2758dbc389ca463eabf66308225cd94f"
format_stamp: "Formatted at 2026-08-06 18:48:46 on dist-test-slave-fgb3"
server_key: "c416b1b3532c1a63d529a9f174f3ca63"
server_key_iv: "483e1e167361633294c4485508d1b11e"
server_key_version: "encryptionkey@0"
I20260806 18:48:46.562466 28411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "2758dbc389ca463eabf66308225cd94f"
format_stamp: "Formatted at 2026-08-06 18:48:46 on dist-test-slave-fgb3"
server_key: "c416b1b3532c1a63d529a9f174f3ca63"
server_key_iv: "483e1e167361633294c4485508d1b11e"
server_key_version: "encryptionkey@0"
I20260806 18:48:46.566624 28411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:48:46.569180 28439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:46.570480 28411 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:46.570609 28411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "2758dbc389ca463eabf66308225cd94f"
format_stamp: "Formatted at 2026-08-06 18:48:46 on dist-test-slave-fgb3"
server_key: "c416b1b3532c1a63d529a9f174f3ca63"
server_key_iv: "483e1e167361633294c4485508d1b11e"
server_key_version: "encryptionkey@0"
I20260806 18:48:46.570715 28411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:46.587604 28411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:46.590884 28411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:46.591068 28411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:46.597931 28411 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:38337
I20260806 18:48:46.597993 28519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:38337 every 8 connection(s)
I20260806 18:48:46.598928 28411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:46.601586 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 28411
I20260806 18:48:46.601719 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:46.601851 28520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:46.602041 2005 external_mini_cluster.cc:1428] Setting key ee3c9b9979063049ff0383db5ed9e049
I20260806 18:48:46.609061 28520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f: Bootstrap starting.
Aug 06 18:48:46 dist-test-slave-fgb3 krb5kdc[28391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:46.611631 28520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:46.612417 28520 log.cc:824] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:46.614454 28520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f: No bootstrap required, opened a new log
I20260806 18:48:46.616446 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:46.603423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36362 (local address 127.1.245.126:38337)
0806 18:48:46.603843 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:46.603861 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:46.603903 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:48:46.604640 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:46.604662 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:46.605087 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:46.605460 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:46.606151 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:46.607629 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:46.608470 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:46.609169 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:46.609340 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:46.611964 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:46.611969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:46.611983 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:46.612014 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:46.613667 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:46.614116 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:46.614120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:46.614125 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:46.614226 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:46.614510 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:46.614514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:46.614516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:46.614927 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:46.615070 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:46.615423 (+ 353us) server_negotiation.cc:299] Negotiation successful
0806 18:48:46.615714 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":123,"threads_started":1}
I20260806 18:48:46.618480 28520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2758dbc389ca463eabf66308225cd94f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38337 } }
I20260806 18:48:46.618714 28520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:46.618789 28520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2758dbc389ca463eabf66308225cd94f, State: Initialized, Role: FOLLOWER
I20260806 18:48:46.619223 28520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2758dbc389ca463eabf66308225cd94f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38337 } }
I20260806 18:48:46.619351 28520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:46.619410 28520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:46.619500 28520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:46.620397 28520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2758dbc389ca463eabf66308225cd94f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38337 } }
I20260806 18:48:46.620713 28520 leader_election.cc:302] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2758dbc389ca463eabf66308225cd94f; no voters:
I20260806 18:48:46.621004 28520 leader_election.cc:288] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:46.621140 28526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:46.621369 28526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [term 1 LEADER]: Becoming Leader. State: Replica: 2758dbc389ca463eabf66308225cd94f, State: Running, Role: LEADER
I20260806 18:48:46.621685 28526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2758dbc389ca463eabf66308225cd94f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38337 } }
I20260806 18:48:46.622218 28520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:46.623271 28527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2758dbc389ca463eabf66308225cd94f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2758dbc389ca463eabf66308225cd94f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38337 } } }
I20260806 18:48:46.623413 28527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:46.623798 28542 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:46.623867 28529 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2758dbc389ca463eabf66308225cd94f. Latest consensus state: current_term: 1 leader_uuid: "2758dbc389ca463eabf66308225cd94f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2758dbc389ca463eabf66308225cd94f" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38337 } } }
I20260806 18:48:46.624006 28529 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:46.626674 28542 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:46.632225 28542 catalog_manager.cc:1347] Generated new cluster ID: 3a423ca4cd59410799710e3b67a0a8cb
I20260806 18:48:46.632299 28542 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:46.657702 28542 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:46.659139 28542 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:46.668880 28542 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f: Generated new TSK 0
I20260806 18:48:46.669812 28542 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:48:46.701642 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38337
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:46.817757 28560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:46.818065 28560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:46.818113 28560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:46.822105 28560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:46.822168 28560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:46.822233 28560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:48:46.827551 28560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:46.828579 28560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:46.829563 28560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:46.839363 28573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:46.839363 28570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:46.839665 28560 server_base.cc:1034] running on GCE node
W20260806 18:48:46.839654 28568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:46.840245 28560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:46.840929 28560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:46.842089 28560 hybrid_clock.cc:648] HybridClock initialized: now 1786042126842061 us; error 40 us; skew 500 ppm
Aug 06 18:48:46 dist-test-slave-fgb3 krb5kdc[28391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042126, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:46.844982 28560 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:48:46.845863 28560 webserver.cc:460] Webserver started at http://127.1.245.65:34753/ using document root <none> and password file <none>
I20260806 18:48:46.846436 28560 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:46.846504 28560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:46.846679 28560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:46.848668 28560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "94156b5076ea4dc4bec9c0ff2d09da52"
format_stamp: "Formatted at 2026-08-06 18:48:46 on dist-test-slave-fgb3"
server_key: "08f50c84f6d7abc21c8783231eff0ece"
server_key_iv: "24c08d194012a9c4164548b935ae3849"
server_key_version: "encryptionkey@0"
I20260806 18:48:46.849179 28560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "94156b5076ea4dc4bec9c0ff2d09da52"
format_stamp: "Formatted at 2026-08-06 18:48:46 on dist-test-slave-fgb3"
server_key: "08f50c84f6d7abc21c8783231eff0ece"
server_key_iv: "24c08d194012a9c4164548b935ae3849"
server_key_version: "encryptionkey@0"
I20260806 18:48:46.852823 28560 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:48:46.855261 28581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:46.856424 28560 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:46.856532 28560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "94156b5076ea4dc4bec9c0ff2d09da52"
format_stamp: "Formatted at 2026-08-06 18:48:46 on dist-test-slave-fgb3"
server_key: "08f50c84f6d7abc21c8783231eff0ece"
server_key_iv: "24c08d194012a9c4164548b935ae3849"
server_key_version: "encryptionkey@0"
I20260806 18:48:46.856638 28560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:46.870846 28560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:46.873734 28560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:46.873940 28560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:46.874594 28560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:46.875461 28560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:46.875509 28560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:46.875552 28560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:46.875581 28560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:46.884361 28560 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:37015
I20260806 18:48:46.884385 28751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:37015 every 8 connection(s)
I20260806 18:48:46.885277 28560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:48:46.889067 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 28560
I20260806 18:48:46.889223 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:48:46.889547 2005 external_mini_cluster.cc:1428] Setting key 22df26aedcfd81e836ada90934d524e4
Aug 06 18:48:46 dist-test-slave-fgb3 krb5kdc[28391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042126, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:46.899544 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:46.887159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:36917 (local address 127.1.245.126:38337)
0806 18:48:46.887341 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:46.887346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:46.888169 (+ 823us) server_negotiation.cc:407] Connection header received
0806 18:48:46.889181 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:46.889186 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:46.889244 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:46.889343 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:46.890716 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:46.891256 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:46.892392 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:46.892718 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:46.892829 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:46.895642 (+ 2813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:46.895646 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:46.895648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:46.895674 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:46.897089 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:46.897715 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:46.897718 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:46.897720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:46.897800 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:46.898255 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:46.898259 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:46.898260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:46.898504 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:46.898594 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:46.899300 (+ 706us) server_negotiation.cc:299] Negotiation successful
0806 18:48:46.899416 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:48:46.900535 28757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:46.887542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38337 (local address 127.1.245.65:36917)
0806 18:48:46.888013 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:46.888058 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:46.888943 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:46.889538 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:46.889546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:46.890024 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:46.890569 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:46.890585 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:46.891381 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:46.891384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:46.892271 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:46.892282 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:46.892885 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:46.892888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:46.893048 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:46.893612 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:46.893628 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:46.895419 (+ 1791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:46.897214 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:46.897221 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:46.897240 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:46.897607 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:46.897885 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:46.897888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:46.897891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:46.898100 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:46.898618 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:46.898625 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:46.898986 (+ 361us) client_negotiation.cc:769] Sending connection context
0806 18:48:46.899226 (+ 240us) client_negotiation.cc:240] Negotiation successful
0806 18:48:46.899597 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":115,"threads_started":1}
I20260806 18:48:46.901935 28754 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38337
I20260806 18:48:46.902222 28754 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:48:46.902879 28754 heartbeater.cc:511] Master 127.1.245.126:38337 requested a full tablet report, sending...
I20260806 18:48:46.904595 28463 ts_manager.cc:194] Registered new tserver with Master: 94156b5076ea4dc4bec9c0ff2d09da52 (127.1.245.65:37015)
I20260806 18:48:46.906023 28463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:36917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:48:46.919512 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38337
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:47.030436 28761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:47.030694 28761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:47.030752 28761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:47.034714 28761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:47.034773 28761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:47.034852 28761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:48:47.040069 28761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:47.041112 28761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:47.042102 28761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:47.051520 28771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:47.051532 28767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:47.051520 28769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:47.051852 28761 server_base.cc:1034] running on GCE node
I20260806 18:48:47.052255 28761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:47.052901 28761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:47.054059 28761 hybrid_clock.cc:648] HybridClock initialized: now 1786042127054037 us; error 30 us; skew 500 ppm
Aug 06 18:48:47 dist-test-slave-fgb3 krb5kdc[28391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:47.056850 28761 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:48:47.057727 28761 webserver.cc:460] Webserver started at http://127.1.245.66:46825/ using document root <none> and password file <none>
I20260806 18:48:47.058346 28761 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:47.058421 28761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:47.058617 28761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:47.060637 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "605f806c167448bfa06631403244f1c2"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "12f90394d81b09a4e9abaac193e44c97"
server_key_iv: "19ffa9a1dce97b740a8be43688523c9c"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.061177 28761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "605f806c167448bfa06631403244f1c2"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "12f90394d81b09a4e9abaac193e44c97"
server_key_iv: "19ffa9a1dce97b740a8be43688523c9c"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.064839 28761 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260806 18:48:47.067209 28782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:47.068411 28761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260806 18:48:47.068531 28761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "605f806c167448bfa06631403244f1c2"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "12f90394d81b09a4e9abaac193e44c97"
server_key_iv: "19ffa9a1dce97b740a8be43688523c9c"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.068640 28761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:47.091162 28761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:47.094131 28761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:47.094390 28761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:47.094990 28761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:47.095850 28761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:47.095909 28761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:47.095968 28761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:47.095992 28761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:47.104176 28761 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:39195
I20260806 18:48:47.104223 28945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:39195 every 8 connection(s)
I20260806 18:48:47.105083 28761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:48:47.106154 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 28761
I20260806 18:48:47.106256 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:48:47.106566 2005 external_mini_cluster.cc:1428] Setting key 38d329bef231238ec38180ebb9ce66bd
Aug 06 18:48:47 dist-test-slave-fgb3 krb5kdc[28391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:48:47.119294 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.106981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:47487 (local address 127.1.245.126:38337)
0806 18:48:47.107108 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.107113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.108104 (+ 991us) server_negotiation.cc:407] Connection header received
0806 18:48:47.109222 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.109225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.109282 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.109358 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:47.110775 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.111561 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.112674 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.113016 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.113128 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.115653 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:47.115658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:47.115660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:47.115692 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:47.117182 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:47.117684 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:47.117689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:47.117690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:47.117769 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:47.118099 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:47.118102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:47.118104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:47.118299 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:47.118429 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.119060 (+ 631us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.119169 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
I20260806 18:48:47.120388 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.107425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38337 (local address 127.1.245.66:47487)
0806 18:48:47.107969 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:47.108013 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:47.108981 (+ 968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:47.109560 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:47.109569 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:47.110087 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:47.110632 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.110646 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.111675 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.111678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.112544 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.112553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.113165 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.113169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.113355 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.113895 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:47.113912 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:47.115482 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:47.117308 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:47.117314 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:47.117335 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:47.117592 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:47.117852 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:47.117855 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:47.117858 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:47.118010 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:47.118465 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:47.118474 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:47.118821 (+ 347us) client_negotiation.cc:769] Sending connection context
0806 18:48:47.119010 (+ 189us) client_negotiation.cc:240] Negotiation successful
0806 18:48:47.119300 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":149,"threads_started":1}
I20260806 18:48:47.121733 28946 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38337
I20260806 18:48:47.121994 28946 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:47.122643 28946 heartbeater.cc:511] Master 127.1.245.126:38337 requested a full tablet report, sending...
I20260806 18:48:47.123940 28463 ts_manager.cc:194] Registered new tserver with Master: 605f806c167448bfa06631403244f1c2 (127.1.245.66:39195)
I20260806 18:48:47.124660 28463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:47487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:48:47.135555 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38337
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:41033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:47.252586 28954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:47.252871 28954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:47.252936 28954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:47.257012 28954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:47.257081 28954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:47.257160 28954 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:48:47.262722 28954 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:41033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38337
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28954
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:47.263880 28954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:47.264917 28954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:47.274857 28960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:47.274897 28964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:47.274817 28962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:47.274996 28954 server_base.cc:1034] running on GCE node
I20260806 18:48:47.275533 28954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:47.276165 28954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:47.277341 28954 hybrid_clock.cc:648] HybridClock initialized: now 1786042127277314 us; error 38 us; skew 500 ppm
Aug 06 18:48:47 dist-test-slave-fgb3 krb5kdc[28391](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:47.280180 28954 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:48:47.281096 28954 webserver.cc:460] Webserver started at http://127.1.245.67:39977/ using document root <none> and password file <none>
I20260806 18:48:47.281656 28954 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:47.281723 28954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:47.281960 28954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:47.284049 28954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "daf67b4bdba4435aac90b8d8e9d359be"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "5850913ec5b6fd58e10f68a61f609a45"
server_key_iv: "a8568b01a624470b1b5b7ff5dc1642bb"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.284582 28954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "daf67b4bdba4435aac90b8d8e9d359be"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "5850913ec5b6fd58e10f68a61f609a45"
server_key_iv: "a8568b01a624470b1b5b7ff5dc1642bb"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.288342 28954 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260806 18:48:47.290830 28974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:47.292142 28954 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:47.292255 28954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "daf67b4bdba4435aac90b8d8e9d359be"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "5850913ec5b6fd58e10f68a61f609a45"
server_key_iv: "a8568b01a624470b1b5b7ff5dc1642bb"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.292361 28954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:47.315310 28954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:47.319111 28954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:47.319303 28954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:47.319929 28954 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:47.320811 28954 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:47.320861 28954 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:47.320919 28954 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:47.320947 28954 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:47.329604 28954 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:35447
I20260806 18:48:47.329629 29126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:35447 every 8 connection(s)
I20260806 18:48:47.330536 28954 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:48:47.332840 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 28954
I20260806 18:48:47.332969 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:48:47.333365 2005 external_mini_cluster.cc:1428] Setting key 727abb14ef9cd772cb25428c354ab06f
Aug 06 18:48:47 dist-test-slave-fgb3 krb5kdc[28391](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042127, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:47.343417 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.332287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:33397 (local address 127.1.245.126:38337)
0806 18:48:47.332413 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.332417 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.333347 (+ 930us) server_negotiation.cc:407] Connection header received
0806 18:48:47.334382 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.334386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.334443 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.334521 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:47.335886 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.336452 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.337563 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.337813 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.337911 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.340358 (+ 2447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:47.340362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:47.340364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:47.340390 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:47.341543 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:47.341994 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:47.341997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:47.341998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:47.342073 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:47.342396 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:47.342399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:47.342400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:47.342588 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:47.342674 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.343131 (+ 457us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.343270 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:48:47.344165 29129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.332650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38337 (local address 127.1.245.67:33397)
0806 18:48:47.333160 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:47.333221 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:47.334152 (+ 931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:47.334725 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:47.334735 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:47.335145 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:47.335722 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.335738 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.336549 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.336551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.337440 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.337450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.337989 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.337993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.338140 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.338705 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:47.338722 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:47.340199 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:47.341642 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:47.341646 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:47.341659 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:47.341915 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:47.342138 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:47.342140 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:47.342142 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:47.342260 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:47.342682 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:47.342687 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:47.342934 (+ 247us) client_negotiation.cc:769] Sending connection context
0806 18:48:47.343087 (+ 153us) client_negotiation.cc:240] Negotiation successful
0806 18:48:47.343339 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":114,"threads_started":1}
I20260806 18:48:47.345201 29127 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38337
I20260806 18:48:47.345427 29127 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:47.345965 29127 heartbeater.cc:511] Master 127.1.245.126:38337 requested a full tablet report, sending...
I20260806 18:48:47.347043 28463 ts_manager.cc:194] Registered new tserver with Master: daf67b4bdba4435aac90b8d8e9d359be (127.1.245.67:35447)
I20260806 18:48:47.347606 28463 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:33397
I20260806 18:48:47.359104 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:48:47.368453 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.361101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36378 (local address 127.1.245.126:38337)
0806 18:48:47.361264 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.361269 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.361409 (+ 140us) server_negotiation.cc:407] Connection header received
0806 18:48:47.361506 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.361509 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.361552 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.361630 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:47.362403 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.362958 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.363824 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.364051 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.364139 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.365651 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:47.365654 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:47.365656 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:47.365680 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:47.367030 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:47.367481 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:47.367484 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:47.367485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:47.367556 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:47.367784 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:47.367786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:47.367787 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:47.367975 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:47.368082 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.368260 (+ 178us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.368334 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260806 18:48:47.371174 28463 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36378:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:48:47.374217 28463 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:48:47.388499 29144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.382679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35447 (local address 127.0.0.1:57562)
0806 18:48:47.383138 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:47.383150 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:47.383226 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:47.384793 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:47.384796 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:47.384816 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:47.384898 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.384904 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.386022 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.386025 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.387198 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.387209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.387972 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.387976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.388079 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.388115 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 18:48:47.388220 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 18:48:47.388342 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":85,"thread_start_us":105,"threads_started":1}
I20260806 18:48:47.388666 29142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.382299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:37015 (local address 127.0.0.1:46564)
0806 18:48:47.382707 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:47.382760 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:47.382903 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:47.383974 (+ 1071us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:47.383980 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:47.384003 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:47.384160 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.384169 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.385601 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.385603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.387171 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.387183 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.388278 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.388282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.388401 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.388426 (+ 25us) client_negotiation.cc:769] Sending connection context
0806 18:48:47.388473 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 18:48:47.388530 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":126,"thread_start_us":87,"threads_started":1}
I20260806 18:48:47.388674 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.383170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57562 (local address 127.1.245.67:35447)
0806 18:48:47.384339 (+ 1169us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.384362 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.384384 (+ 22us) server_negotiation.cc:407] Connection header received
0806 18:48:47.384456 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.384460 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.384631 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.384808 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:47.384995 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.385905 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.387311 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.387858 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.387961 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.388117 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.388446 (+ 329us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.388544 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1065,"thread_start_us":144,"threads_started":1}
I20260806 18:48:47.389285 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.382508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46564 (local address 127.1.245.65:37015)
0806 18:48:47.383536 (+ 1028us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.383552 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.383604 (+ 52us) server_negotiation.cc:407] Connection header received
0806 18:48:47.383669 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.383673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.383829 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.383972 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:47.384300 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.385483 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.387318 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.388139 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.388849 (+ 710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.389000 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.389050 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.389161 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":926,"thread_start_us":82,"threads_started":1}
I20260806 18:48:47.389736 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.383522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43526 (local address 127.1.245.66:39195)
0806 18:48:47.383827 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.383843 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.383869 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:48:47.383945 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.383950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.384191 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.384346 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:47.384571 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.385432 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.387342 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.388139 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.388250 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.388401 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.389499 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.389598 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":98,"threads_started":1}
I20260806 18:48:47.389900 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.383099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:39195 (local address 127.0.0.1:43526)
0806 18:48:47.383317 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:47.383329 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:47.383401 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:47.384341 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:47.384344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:47.384361 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:47.384460 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.384466 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.385549 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.385552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.387217 (+ 1665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.387225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.388257 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.388260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.388340 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.389381 (+ 1041us) client_negotiation.cc:769] Sending connection context
0806 18:48:47.389458 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 18:48:47.389546 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":1102,"thread_start_us":99,"threads_started":1}
I20260806 18:48:47.391788 29039 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b (DEFAULT_TABLE table=test-table [id=a2286f39d70e426fafb4a2e9e197beeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:47.392299 28654 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b (DEFAULT_TABLE table=test-table [id=a2286f39d70e426fafb4a2e9e197beeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:47.393015 29039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:47.393211 28654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:47.392815 28852 tablet_service.cc:1467] Processing CreateTablet for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b (DEFAULT_TABLE table=test-table [id=a2286f39d70e426fafb4a2e9e197beeb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:47.393769 28852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:47.400166 29152 tablet_bootstrap.cc:492] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: Bootstrap starting.
I20260806 18:48:47.400700 29151 tablet_bootstrap.cc:492] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52: Bootstrap starting.
I20260806 18:48:47.401132 29154 tablet_bootstrap.cc:492] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: Bootstrap starting.
I20260806 18:48:47.402696 29152 tablet_bootstrap.cc:654] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:47.402828 29151 tablet_bootstrap.cc:654] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:47.403421 29154 tablet_bootstrap.cc:654] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:47.403761 29152 log.cc:824] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:47.403784 29151 log.cc:824] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:47.404163 29154 log.cc:824] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:47.405758 29152 tablet_bootstrap.cc:492] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: No bootstrap required, opened a new log
I20260806 18:48:47.405970 29152 ts_tablet_manager.cc:1397] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:48:47.406044 29154 tablet_bootstrap.cc:492] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: No bootstrap required, opened a new log
I20260806 18:48:47.406204 29151 tablet_bootstrap.cc:492] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52: No bootstrap required, opened a new log
I20260806 18:48:47.406217 29154 ts_tablet_manager.cc:1397] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:48:47.406409 29151 ts_tablet_manager.cc:1397] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:48:47.409564 29151 raft_consensus.cc:348] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.409785 29151 raft_consensus.cc:374] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:47.409871 29151 raft_consensus.cc:729] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94156b5076ea4dc4bec9c0ff2d09da52, State: Initialized, Role: FOLLOWER
I20260806 18:48:47.409935 29154 raft_consensus.cc:348] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.409936 29152 raft_consensus.cc:348] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.410203 29154 raft_consensus.cc:374] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:47.410203 29152 raft_consensus.cc:374] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:47.410262 29152 raft_consensus.cc:729] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: daf67b4bdba4435aac90b8d8e9d359be, State: Initialized, Role: FOLLOWER
I20260806 18:48:47.410301 29154 raft_consensus.cc:729] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 605f806c167448bfa06631403244f1c2, State: Initialized, Role: FOLLOWER
I20260806 18:48:47.410377 29151 consensus_queue.cc:260] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.410723 29152 consensus_queue.cc:260] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.410723 29154 consensus_queue.cc:260] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.411176 28754 heartbeater.cc:503] Master 127.1.245.126:38337 was elected leader, sending a full tablet report...
I20260806 18:48:47.411569 29151 ts_tablet_manager.cc:1428] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:48:47.411700 29152 ts_tablet_manager.cc:1428] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260806 18:48:47.411700 29154 ts_tablet_manager.cc:1428] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260806 18:48:47.412019 28946 heartbeater.cc:503] Master 127.1.245.126:38337 was elected leader, sending a full tablet report...
I20260806 18:48:47.412963 29127 heartbeater.cc:503] Master 127.1.245.126:38337 was elected leader, sending a full tablet report...
I20260806 18:48:47.478371 29162 raft_consensus.cc:482] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:47.478591 29162 raft_consensus.cc:504] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.479790 29162 leader_election.cc:288] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94156b5076ea4dc4bec9c0ff2d09da52 (127.1.245.65:37015), daf67b4bdba4435aac90b8d8e9d359be (127.1.245.67:35447)
I20260806 18:48:47.483650 28949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.480063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:35447 (local address 127.1.245.66:43151)
0806 18:48:47.480309 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:47.480325 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:47.480428 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:47.480802 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:47.480806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:47.480829 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:47.480950 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.480958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.481733 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.481736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.482573 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.482583 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.483240 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.483244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.483348 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.483367 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:48:47.483431 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:48:47.483505 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260806 18:48:47.483650 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.480213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:43151 (local address 127.1.245.67:35447)
0806 18:48:47.480346 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.480350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.480371 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:48:47.480550 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.480553 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.480605 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.480684 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:47.481083 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.481624 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.482698 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.483117 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.483232 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.483290 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.483464 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.483541 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:48:47.483809 29143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.480271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:39051 (local address 127.1.245.65:37015)
0806 18:48:47.480386 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.480390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.480877 (+ 487us) server_negotiation.cc:407] Connection header received
0806 18:48:47.480978 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.480981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.481022 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.481080 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:47.481345 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.481841 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.482913 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.483305 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.483409 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.483460 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.483627 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.483696 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:48:47.483817 29164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.480063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:37015 (local address 127.1.245.66:39051)
0806 18:48:47.480788 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:47.480801 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:47.480894 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:47.481099 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:47.481102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:47.481116 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:47.481237 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.481243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.481940 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.481943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.482793 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:47.482803 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.483409 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.483413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:47.483523 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.483541 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:48:47.483604 (+ 63us) client_negotiation.cc:240] Negotiation successful
0806 18:48:47.483671 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":337,"thread_start_us":151,"threads_started":1}
I20260806 18:48:47.484447 28685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2dbadf55d54d49b1ded3cdf09fcd5b" candidate_uuid: "605f806c167448bfa06631403244f1c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" is_pre_election: true
I20260806 18:48:47.484447 29064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2dbadf55d54d49b1ded3cdf09fcd5b" candidate_uuid: "605f806c167448bfa06631403244f1c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf67b4bdba4435aac90b8d8e9d359be" is_pre_election: true
I20260806 18:48:47.484747 28685 raft_consensus.cc:2454] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 605f806c167448bfa06631403244f1c2 in term 0.
I20260806 18:48:47.484756 29064 raft_consensus.cc:2454] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 605f806c167448bfa06631403244f1c2 in term 0.
I20260806 18:48:47.485265 28784 leader_election.cc:302] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 605f806c167448bfa06631403244f1c2, 94156b5076ea4dc4bec9c0ff2d09da52; no voters:
I20260806 18:48:47.485566 29162 raft_consensus.cc:2781] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:48:47.485649 29162 raft_consensus.cc:482] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:48:47.485679 29162 raft_consensus.cc:3037] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:47.486807 29162 raft_consensus.cc:504] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.487151 29162 leader_election.cc:288] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [CANDIDATE]: Term 1 election: Requested vote from peers 94156b5076ea4dc4bec9c0ff2d09da52 (127.1.245.65:37015), daf67b4bdba4435aac90b8d8e9d359be (127.1.245.67:35447)
I20260806 18:48:47.487576 28685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2dbadf55d54d49b1ded3cdf09fcd5b" candidate_uuid: "605f806c167448bfa06631403244f1c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94156b5076ea4dc4bec9c0ff2d09da52"
I20260806 18:48:47.487628 29064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3c2dbadf55d54d49b1ded3cdf09fcd5b" candidate_uuid: "605f806c167448bfa06631403244f1c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "daf67b4bdba4435aac90b8d8e9d359be"
I20260806 18:48:47.487696 28685 raft_consensus.cc:3037] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:47.487753 29064 raft_consensus.cc:3037] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:47.488803 29064 raft_consensus.cc:2454] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 605f806c167448bfa06631403244f1c2 in term 1.
I20260806 18:48:47.488832 28685 raft_consensus.cc:2454] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 605f806c167448bfa06631403244f1c2 in term 1.
I20260806 18:48:47.489111 28787 leader_election.cc:302] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 605f806c167448bfa06631403244f1c2, daf67b4bdba4435aac90b8d8e9d359be; no voters:
I20260806 18:48:47.489395 29162 raft_consensus.cc:2781] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:47.489692 29162 raft_consensus.cc:686] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 1 LEADER]: Becoming Leader. State: Replica: 605f806c167448bfa06631403244f1c2, State: Running, Role: LEADER
I20260806 18:48:47.490077 29162 consensus_queue.cc:237] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } }
I20260806 18:48:47.493669 28463 catalog_manager.cc:5495] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 605f806c167448bfa06631403244f1c2 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "605f806c167448bfa06631403244f1c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "605f806c167448bfa06631403244f1c2" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 39195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 } health_report { overall_health: UNKNOWN } } }
I20260806 18:48:47.513685 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.510060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.1.245.66:39195)
0806 18:48:47.510183 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.510186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.510202 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:48:47.510341 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.510345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.510412 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.510510 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:48:47.510829 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.511385 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.512264 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.512498 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.512586 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.512852 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:48:47.513262 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:48:47.513406 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.513511 (+ 105us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.513548 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:48:47.519610 29064 raft_consensus.cc:1261] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 1 FOLLOWER]: Refusing update from remote peer 605f806c167448bfa06631403244f1c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:47.519650 28685 raft_consensus.cc:1261] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 1 FOLLOWER]: Refusing update from remote peer 605f806c167448bfa06631403244f1c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:47.520342 29165 consensus_queue.cc:1035] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "daf67b4bdba4435aac90b8d8e9d359be" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 35447 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:47.520532 29162 consensus_queue.cc:1035] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "94156b5076ea4dc4bec9c0ff2d09da52" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:47.536538 28463 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:47.536732 28463 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36378:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:47.539283 28463 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2758dbc389ca463eabf66308225cd94f: Sending DeleteTablet for 3 replicas of tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b
I20260806 18:48:47.540186 28654 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:46564
I20260806 18:48:47.540371 29039 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:57562
I20260806 18:48:47.540874 29186 tablet_replica.cc:330] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: stopping tablet replica
I20260806 18:48:47.540913 29185 tablet_replica.cc:330] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52: stopping tablet replica
I20260806 18:48:47.540855 28852 tablet_service.cc:1514] Processing DeleteTablet for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:47 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:43526
I20260806 18:48:47.541297 29185 raft_consensus.cc:2229] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:47.541313 29186 raft_consensus.cc:2229] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:47.541343 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 28560
I20260806 18:48:47.541679 29186 raft_consensus.cc:2258] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:47.541682 29185 raft_consensus.cc:2258] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 94156b5076ea4dc4bec9c0ff2d09da52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:47.547009 29186 ts_tablet_manager.cc:1905] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:47.550943 29186 ts_tablet_manager.cc:1918] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:47.551373 29188 tablet_replica.cc:330] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: stopping tablet replica
I20260806 18:48:47.551934 29188 raft_consensus.cc:2229] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:48:47.552624 29188 raft_consensus.cc:2258] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:47.553344 29186 log.cc:1196] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/3c2dbadf55d54d49b1ded3cdf09fcd5b
I20260806 18:48:47.555364 29188 ts_tablet_manager.cc:1905] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:47.555974 29186 ts_tablet_manager.cc:1939] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P daf67b4bdba4435aac90b8d8e9d359be: Deleting consensus metadata
I20260806 18:48:47.557828 28444 catalog_manager.cc:4841] TS daf67b4bdba4435aac90b8d8e9d359be (127.1.245.67:35447): tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b (table test-table [id=a2286f39d70e426fafb4a2e9e197beeb]) successfully deleted
I20260806 18:48:47.561585 29188 ts_tablet_manager.cc:1918] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:47.561945 29188 log.cc:1196] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/3c2dbadf55d54d49b1ded3cdf09fcd5b
I20260806 18:48:47.562497 29188 ts_tablet_manager.cc:1939] T 3c2dbadf55d54d49b1ded3cdf09fcd5b P 605f806c167448bfa06631403244f1c2: Deleting consensus metadata
I20260806 18:48:47.565130 28442 catalog_manager.cc:4841] TS 605f806c167448bfa06631403244f1c2 (127.1.245.66:39195): tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b (table test-table [id=a2286f39d70e426fafb4a2e9e197beeb]) successfully deleted
W20260806 18:48:47.572619 28442 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:37015 (error 108)
I20260806 18:48:47.573459 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 28761
I20260806 18:48:47.573794 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.573470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:37015 (local address 127.0.0.1:46568)
0806 18:48:47.573610 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:47.573697 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:37015: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260806 18:48:47.574066 28442 catalog_manager.cc:4584] TS 94156b5076ea4dc4bec9c0ff2d09da52 (127.1.245.65:37015): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3c2dbadf55d54d49b1ded3cdf09fcd5b: Network error: Client connection negotiation failed: client connection to 127.1.245.65:37015: connect: Connection refused (error 111)
I20260806 18:48:47.584337 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 28954
I20260806 18:48:47.596419 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 28411
2026-08-06T18:48:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2432 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T18:48:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:47Z Disabled control of system clock
I20260806 18:48:47.630605 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:42115
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:46359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:42115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:48:47.744454 29195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:47.744741 29195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:47.744778 29195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:47.748873 29195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:47.748948 29195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:47.748962 29195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:47.748978 29195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:47.748991 29195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:47.753986 29195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:42115
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:42115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:47.755225 29195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:47.756687 29195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:47.765370 29208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:47.765365 29205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:47.765365 29206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:47.765413 29195 server_base.cc:1034] running on GCE node
I20260806 18:48:47.766115 29195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:47.767225 29195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:47.768395 29195 hybrid_clock.cc:648] HybridClock initialized: now 1786042127768378 us; error 33 us; skew 500 ppm
I20260806 18:48:47.770462 29195 webserver.cc:460] Webserver started at http://127.1.245.126:35101/ using document root <none> and password file <none>
I20260806 18:48:47.771046 29195 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:47.771111 29195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:47.771325 29195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:47.773367 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "f47056ddb1f347d5b91129dd3d9cd7ff"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "eeb5ada4959abf8cbc6fa527ff789e64"
server_key_iv: "c657caa65a28b3dc93bce2dcd0d9cddd"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.773904 29195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "f47056ddb1f347d5b91129dd3d9cd7ff"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "eeb5ada4959abf8cbc6fa527ff789e64"
server_key_iv: "c657caa65a28b3dc93bce2dcd0d9cddd"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.777580 29195 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 18:48:47.780233 29215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:47.781555 29195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260806 18:48:47.781704 29195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "f47056ddb1f347d5b91129dd3d9cd7ff"
format_stamp: "Formatted at 2026-08-06 18:48:47 on dist-test-slave-fgb3"
server_key: "eeb5ada4959abf8cbc6fa527ff789e64"
server_key_iv: "c657caa65a28b3dc93bce2dcd0d9cddd"
server_key_version: "encryptionkey@0"
I20260806 18:48:47.781814 29195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:47.796011 29195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:47.796691 29195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:47.796866 29195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:47.803735 29195 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:42115
I20260806 18:48:47.803756 29288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:42115 every 8 connection(s)
I20260806 18:48:47.804664 29195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:47.806612 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 29195
I20260806 18:48:47.806728 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:47.807044 2005 external_mini_cluster.cc:1428] Setting key c49f878ebfb095a696458f0dd552b44e
I20260806 18:48:47.808300 29289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:47.813915 29289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff: Bootstrap starting.
I20260806 18:48:47.815950 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:47.808467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.1.245.126:42115)
0806 18:48:47.809098 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:47.809113 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:47.809148 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:48:47.809782 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:47.809798 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:47.810171 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:47.810545 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:47.810966 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.811919 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.812779 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:47.813256 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:47.813413 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:47.814235 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:47.814240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:47.814252 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:47.814683 (+ 431us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:47.814753 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:47.814773 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:47.814864 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:47.814990 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 18:48:47.815248 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":150,"threads_started":1}
I20260806 18:48:47.816203 29289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:47.816967 29289 log.cc:824] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:47.818975 29289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff: No bootstrap required, opened a new log
I20260806 18:48:47.822564 29289 raft_consensus.cc:348] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47056ddb1f347d5b91129dd3d9cd7ff" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42115 } }
I20260806 18:48:47.822781 29289 raft_consensus.cc:374] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:47.822829 29289 raft_consensus.cc:729] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f47056ddb1f347d5b91129dd3d9cd7ff, State: Initialized, Role: FOLLOWER
I20260806 18:48:47.823179 29289 consensus_queue.cc:260] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47056ddb1f347d5b91129dd3d9cd7ff" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42115 } }
I20260806 18:48:47.823292 29289 raft_consensus.cc:388] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:47.823338 29289 raft_consensus.cc:482] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:47.823405 29289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:47.824214 29289 raft_consensus.cc:504] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47056ddb1f347d5b91129dd3d9cd7ff" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42115 } }
I20260806 18:48:47.824501 29289 leader_election.cc:302] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f47056ddb1f347d5b91129dd3d9cd7ff; no voters:
I20260806 18:48:47.824754 29289 leader_election.cc:288] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:47.824848 29294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:47.825112 29294 raft_consensus.cc:686] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [term 1 LEADER]: Becoming Leader. State: Replica: f47056ddb1f347d5b91129dd3d9cd7ff, State: Running, Role: LEADER
I20260806 18:48:47.825431 29294 consensus_queue.cc:237] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47056ddb1f347d5b91129dd3d9cd7ff" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42115 } }
I20260806 18:48:47.825891 29289 sys_catalog.cc:564] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:47.826859 29297 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader f47056ddb1f347d5b91129dd3d9cd7ff. Latest consensus state: current_term: 1 leader_uuid: "f47056ddb1f347d5b91129dd3d9cd7ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47056ddb1f347d5b91129dd3d9cd7ff" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42115 } } }
I20260806 18:48:47.826987 29297 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:47.827332 29295 sys_catalog.cc:455] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f47056ddb1f347d5b91129dd3d9cd7ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f47056ddb1f347d5b91129dd3d9cd7ff" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42115 } } }
I20260806 18:48:47.827426 29295 sys_catalog.cc:458] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:47.827514 29309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:47.830178 29309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:47.835088 29309 catalog_manager.cc:1347] Generated new cluster ID: 7beb9e0691044ccca5f5bbe6ac8ba3f6
I20260806 18:48:47.835153 29309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:47.856818 29309 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:47.857795 29309 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:47.866129 29309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff: Generated new TSK 0
I20260806 18:48:47.867022 29309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:48:47.877209 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42115
--builtin_ntp_servers=127.1.245.84:46359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:48:47.992076 29324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:47.992338 29324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:47.992404 29324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:47.996572 29324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:47.996635 29324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:47.996712 29324 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:48:48.001609 29324 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:48.002799 29324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:48.004323 29324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:48.014695 29333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:48.014845 29324 server_base.cc:1034] running on GCE node
W20260806 18:48:48.014763 29331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:48.014695 29336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:48.015601 29324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:48.016337 29324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:48.017522 29324 hybrid_clock.cc:648] HybridClock initialized: now 1786042128017499 us; error 35 us; skew 500 ppm
I20260806 18:48:48.019629 29324 webserver.cc:460] Webserver started at http://127.1.245.65:36855/ using document root <none> and password file <none>
I20260806 18:48:48.020406 29324 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:48.020498 29324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:48.020756 29324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:48.023165 29324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "9437c62f856a412fb211229a40e7cd8d"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "7b61d5174b0289a651fae77145b825fe"
server_key_iv: "9cc5fcc507d94c67e4de140dd1fefd47"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.023813 29324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "9437c62f856a412fb211229a40e7cd8d"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "7b61d5174b0289a651fae77145b825fe"
server_key_iv: "9cc5fcc507d94c67e4de140dd1fefd47"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.028178 29324 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:48:48.030917 29343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.032262 29324 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:48:48.032408 29324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "9437c62f856a412fb211229a40e7cd8d"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "7b61d5174b0289a651fae77145b825fe"
server_key_iv: "9cc5fcc507d94c67e4de140dd1fefd47"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.032526 29324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:48.056203 29324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:48.057009 29324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:48.057229 29324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:48.057898 29324 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:48.059053 29324 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:48.059109 29324 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.059172 29324 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:48.059207 29324 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.069116 29324 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:44661
I20260806 18:48:48.069144 29511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:44661 every 8 connection(s)
I20260806 18:48:48.070142 29324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:48:48.074949 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 29324
I20260806 18:48:48.075074 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:48:48.075331 2005 external_mini_cluster.cc:1428] Setting key 514bff3d6128a38c7bd0cd5b6f920fd4
I20260806 18:48:48.077256 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42115
--builtin_ntp_servers=127.1.245.84:46359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:48:48.079931 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.072079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:48745 (local address 127.1.245.126:42115)
0806 18:48:48.072266 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.072271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.073502 (+ 1231us) server_negotiation.cc:407] Connection header received
0806 18:48:48.074930 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.074934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.075002 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.075077 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:48.076154 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.076752 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.077994 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.078235 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.078346 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.079438 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:48.079442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:48.079444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:48.079459 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:48.079479 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:48.079487 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:48.079546 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.079694 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.079772 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260806 18:48:48.080963 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.072503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:42115 (local address 127.1.245.65:48745)
0806 18:48:48.073333 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.073381 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:48.074695 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:48.075266 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:48.075272 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:48.075402 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:48.076010 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.076027 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.076884 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.076888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.077867 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.077877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.078416 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.078420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.078574 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.079165 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:48.079167 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:48.079210 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:48.079277 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:48.079557 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:48.079566 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:48.079573 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:48:48.079685 (+ 112us) client_negotiation.cc:240] Negotiation successful
0806 18:48:48.079971 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":132,"threads_started":1}
I20260806 18:48:48.082238 29512 heartbeater.cc:347] Connected to a master server at 127.1.245.126:42115
I20260806 18:48:48.082511 29512 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:48.083078 29512 heartbeater.cc:511] Master 127.1.245.126:42115 requested a full tablet report, sending...
I20260806 18:48:48.084700 29241 ts_manager.cc:194] Registered new tserver with Master: 9437c62f856a412fb211229a40e7cd8d (127.1.245.65:44661)
I20260806 18:48:48.085922 29241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.65:48745
W20260806 18:48:48.192699 29516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:48.192965 29516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:48.193004 29516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:48.197009 29516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:48.197067 29516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:48.197124 29516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:48:48.201823 29516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29516
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:48.202950 29516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:48.204330 29516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:48:48.214151 29516 server_base.cc:1034] running on GCE node
W20260806 18:48:48.214097 29529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:48.214093 29523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:48.214372 29525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:48.214850 29516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:48.215515 29516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:48.216687 29516 hybrid_clock.cc:648] HybridClock initialized: now 1786042128216668 us; error 30 us; skew 500 ppm
I20260806 18:48:48.218612 29516 webserver.cc:460] Webserver started at http://127.1.245.66:45687/ using document root <none> and password file <none>
I20260806 18:48:48.219194 29516 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:48.219264 29516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:48.219462 29516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:48.221556 29516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "695b59b26b9c44a1bb5142c96364b5cc"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "92a3671fd2a7389003f7a9c1a0565d84"
server_key_iv: "edc89b319f2d853c93a0a9aabf5ef490"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.222100 29516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "695b59b26b9c44a1bb5142c96364b5cc"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "92a3671fd2a7389003f7a9c1a0565d84"
server_key_iv: "edc89b319f2d853c93a0a9aabf5ef490"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.226011 29516 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:48:48.228686 29538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.229969 29516 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:48:48.230085 29516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "695b59b26b9c44a1bb5142c96364b5cc"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "92a3671fd2a7389003f7a9c1a0565d84"
server_key_iv: "edc89b319f2d853c93a0a9aabf5ef490"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.230191 29516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:48.244980 29516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:48.245690 29516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:48.245877 29516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:48.246513 29516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:48.247489 29516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:48.247551 29516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.247607 29516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:48.247632 29516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.256072 29516 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:45751
I20260806 18:48:48.256104 29709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:45751 every 8 connection(s)
I20260806 18:48:48.257011 29516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:48:48.265143 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 29516
I20260806 18:48:48.265281 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:48:48.265563 2005 external_mini_cluster.cc:1428] Setting key b8894d35f88d12ba29dd83eb8a7c77ae
I20260806 18:48:48.266050 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.258815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:33103 (local address 127.1.245.126:42115)
0806 18:48:48.258950 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.258954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.259989 (+ 1035us) server_negotiation.cc:407] Connection header received
0806 18:48:48.261405 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.261408 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.261466 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.261540 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:48.262465 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.263001 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.264137 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.264369 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.264462 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.265551 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:48.265555 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:48.265558 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:48.265573 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:48.265595 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:48.265603 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:48.265662 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.265822 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.265895 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:48:48.267055 29713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.259206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:42115 (local address 127.1.245.66:33103)
0806 18:48:48.259842 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.259885 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:48.261161 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:48.261721 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:48.261728 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:48.261811 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:48.262285 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.262297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.263122 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.263125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.264012 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.264023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.264516 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.264521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.264676 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.265259 (+ 583us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:48.265261 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:48.265325 (+ 64us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:48.265405 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:48.265672 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:48.265682 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:48.265689 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:48:48.265804 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 18:48:48.266086 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
I20260806 18:48:48.267820 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42115
--builtin_ntp_servers=127.1.245.84:46359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:48:48.268198 29710 heartbeater.cc:347] Connected to a master server at 127.1.245.126:42115
I20260806 18:48:48.268437 29710 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:48.268976 29710 heartbeater.cc:511] Master 127.1.245.126:42115 requested a full tablet report, sending...
I20260806 18:48:48.270048 29241 ts_manager.cc:194] Registered new tserver with Master: 695b59b26b9c44a1bb5142c96364b5cc (127.1.245.66:45751)
I20260806 18:48:48.270666 29241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.66:33103
W20260806 18:48:48.382359 29714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:48.382613 29714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:48.382651 29714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:48.386691 29714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:48.386752 29714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:48.386816 29714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:48:48.391552 29714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:42115
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:48.392678 29714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:48.394098 29714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:48.403580 29726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:48.403620 29723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:48.403620 29722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:48.404035 29714 server_base.cc:1034] running on GCE node
I20260806 18:48:48.404423 29714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:48.405041 29714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:48.406260 29714 hybrid_clock.cc:648] HybridClock initialized: now 1786042128406233 us; error 37 us; skew 500 ppm
I20260806 18:48:48.408231 29714 webserver.cc:460] Webserver started at http://127.1.245.67:42027/ using document root <none> and password file <none>
I20260806 18:48:48.408798 29714 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:48.408865 29714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:48.409049 29714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:48.411296 29714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "feb3e441680943619a1ce237070a838c"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "baadcb7a7a32de7bb9cfbd9f9e2c9a92"
server_key_iv: "935a1c5a37e00214d90471a315372fde"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.411855 29714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "feb3e441680943619a1ce237070a838c"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "baadcb7a7a32de7bb9cfbd9f9e2c9a92"
server_key_iv: "935a1c5a37e00214d90471a315372fde"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.415895 29714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260806 18:48:48.418507 29735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.419873 29714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260806 18:48:48.420007 29714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "feb3e441680943619a1ce237070a838c"
format_stamp: "Formatted at 2026-08-06 18:48:48 on dist-test-slave-fgb3"
server_key: "baadcb7a7a32de7bb9cfbd9f9e2c9a92"
server_key_iv: "935a1c5a37e00214d90471a315372fde"
server_key_version: "encryptionkey@0"
I20260806 18:48:48.420115 29714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:48.434141 29714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:48.434996 29714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:48.435165 29714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:48.435775 29714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:48.436839 29714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:48.436892 29714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.436947 29714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:48.436975 29714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:48.445760 29714 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:39435
I20260806 18:48:48.445780 29914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:39435 every 8 connection(s)
I20260806 18:48:48.446779 29714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:48:48.454962 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 29714
I20260806 18:48:48.455096 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:48:48.455402 2005 external_mini_cluster.cc:1428] Setting key 9087e1505018f45193e597b5b406b0b8
I20260806 18:48:48.456528 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.448679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:47591 (local address 127.1.245.126:42115)
0806 18:48:48.448812 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.448816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.449923 (+ 1107us) server_negotiation.cc:407] Connection header received
0806 18:48:48.451392 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.451395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.451459 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.451543 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:48.452477 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.453006 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.454156 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.454423 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.454518 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.455772 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:48.455777 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:48.455779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:48.455797 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:48.455831 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:48.455843 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:48.455929 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.456280 (+ 351us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.456382 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:48:48.457476 29919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.449051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:42115 (local address 127.1.245.67:47591)
0806 18:48:48.449760 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.449803 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:48.451161 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:48.451741 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:48.451748 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:48.451834 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:48.452325 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.452338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.453129 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.453132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.454027 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.454037 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.454573 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.454577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.454769 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.455452 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:48.455455 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:48.455510 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:48.455595 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:48.456118 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:48.456126 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:48.456131 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 18:48:48.456252 (+ 121us) client_negotiation.cc:240] Negotiation successful
0806 18:48:48.456556 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":121,"threads_started":1}
I20260806 18:48:48.458635 29917 heartbeater.cc:347] Connected to a master server at 127.1.245.126:42115
I20260806 18:48:48.458880 29917 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:48.459437 29917 heartbeater.cc:511] Master 127.1.245.126:42115 requested a full tablet report, sending...
I20260806 18:48:48.460525 29241 ts_manager.cc:194] Registered new tserver with Master: feb3e441680943619a1ce237070a838c (127.1.245.67:39435)
I20260806 18:48:48.461084 29241 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.67:47591
I20260806 18:48:48.469880 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:48:48.475973 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.472014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.1.245.126:42115)
0806 18:48:48.472179 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.472188 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.472269 (+ 81us) server_negotiation.cc:407] Connection header received
0806 18:48:48.472449 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.472452 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.472499 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.472564 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:48.472851 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.473377 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.474202 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.474494 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.474581 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.475476 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:48.475479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:48.475481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:48.475494 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:48.475515 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:48.475523 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:48.475630 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.475781 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.475822 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260806 18:48:48.478926 29241 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:48:48.481943 29241 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:48:48.496817 29938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.491332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38776 (local address 127.1.245.65:44661)
0806 18:48:48.491776 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.491788 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.491814 (+ 26us) server_negotiation.cc:407] Connection header received
0806 18:48:48.491899 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.491904 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.492117 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.492270 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:48.492749 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.493597 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.495261 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.496077 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.496190 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.496395 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.496530 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.496630 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":141,"threads_started":1}
I20260806 18:48:48.496819 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.490992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:44661 (local address 127.0.0.1:38776)
0806 18:48:48.491610 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.491624 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:48.491713 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:48.492455 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:48.492458 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:48.492478 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:48.492621 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.492627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.493694 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.493697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.495113 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.495124 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.496213 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.496218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.496333 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.496381 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 18:48:48.496490 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 18:48:48.496655 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"mutex_wait_us":32,"thread_start_us":88,"threads_started":1}
I20260806 18:48:48.497100 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.490757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37492 (local address 127.1.245.66:45751)
0806 18:48:48.492923 (+ 2166us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.492934 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.492958 (+ 24us) server_negotiation.cc:407] Connection header received
0806 18:48:48.493031 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.493035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.493212 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.493357 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:48.493574 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.494476 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.495686 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.496231 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.496615 (+ 384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.496799 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.496850 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.496944 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2051,"thread_start_us":141,"threads_started":1}
I20260806 18:48:48.497386 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.491378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:39435 (local address 127.0.0.1:58138)
0806 18:48:48.491939 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.491951 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:48.492029 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:48.492422 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:48.492427 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:48.492449 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:48.492611 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.492617 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.493948 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.493951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.495116 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.495124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.497011 (+ 1887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.497015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.497098 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.497121 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:48:48.497195 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 18:48:48.497264 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":29,"spinlock_wait_cycles":768,"thread_start_us":101,"threads_started":1}
I20260806 18:48:48.496819 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.490640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:45751 (local address 127.0.0.1:37492)
0806 18:48:48.491022 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.491075 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:48.491216 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:48.493366 (+ 2150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:48.493370 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:48.493390 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:48.493472 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.493477 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.494610 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.494613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.495554 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.495565 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.496357 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.496361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.496455 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.496475 (+ 20us) client_negotiation.cc:769] Sending connection context
0806 18:48:48.496518 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 18:48:48.496669 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":45,"thread_start_us":102,"threads_started":1}
I20260806 18:48:48.497617 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.491571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.1.245.67:39435)
0806 18:48:48.492007 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.492018 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.492041 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:48:48.492117 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.492122 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.492287 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.492430 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:48.492726 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.493823 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.495261 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.496077 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.496190 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.496395 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.497363 (+ 968us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.497478 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":134,"threads_started":1}
I20260806 18:48:48.500207 29603 tablet_service.cc:1467] Processing CreateTablet for tablet 05e24b30e3a342e5a5c74c88dfbee7d8 (DEFAULT_TABLE table=test-table [id=a22399dde4334f5a89a522a222859aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:48.500205 29416 tablet_service.cc:1467] Processing CreateTablet for tablet 05e24b30e3a342e5a5c74c88dfbee7d8 (DEFAULT_TABLE table=test-table [id=a22399dde4334f5a89a522a222859aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:48.501137 29603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e24b30e3a342e5a5c74c88dfbee7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:48.501137 29416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e24b30e3a342e5a5c74c88dfbee7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:48.504112 29814 tablet_service.cc:1467] Processing CreateTablet for tablet 05e24b30e3a342e5a5c74c88dfbee7d8 (DEFAULT_TABLE table=test-table [id=a22399dde4334f5a89a522a222859aa3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:48.505241 29814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 05e24b30e3a342e5a5c74c88dfbee7d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:48.507943 29940 tablet_bootstrap.cc:492] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: Bootstrap starting.
I20260806 18:48:48.509078 29941 tablet_bootstrap.cc:492] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d: Bootstrap starting.
I20260806 18:48:48.510421 29940 tablet_bootstrap.cc:654] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:48.511250 29941 tablet_bootstrap.cc:654] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:48.511334 29940 log.cc:824] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:48.511785 29942 tablet_bootstrap.cc:492] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: Bootstrap starting.
I20260806 18:48:48.512087 29941 log.cc:824] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:48.513605 29940 tablet_bootstrap.cc:492] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: No bootstrap required, opened a new log
I20260806 18:48:48.513806 29940 ts_tablet_manager.cc:1397] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260806 18:48:48.514214 29942 tablet_bootstrap.cc:654] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:48.514246 29941 tablet_bootstrap.cc:492] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d: No bootstrap required, opened a new log
I20260806 18:48:48.514453 29941 ts_tablet_manager.cc:1397] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:48:48.515103 29942 log.cc:824] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:48.517133 29942 tablet_bootstrap.cc:492] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: No bootstrap required, opened a new log
I20260806 18:48:48.517339 29942 ts_tablet_manager.cc:1397] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:48:48.517346 29940 raft_consensus.cc:348] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.517614 29940 raft_consensus.cc:374] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:48.517668 29940 raft_consensus.cc:729] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 695b59b26b9c44a1bb5142c96364b5cc, State: Initialized, Role: FOLLOWER
I20260806 18:48:48.518111 29940 consensus_queue.cc:260] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.518579 29941 raft_consensus.cc:348] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.518821 29710 heartbeater.cc:503] Master 127.1.245.126:42115 was elected leader, sending a full tablet report...
I20260806 18:48:48.518847 29941 raft_consensus.cc:374] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:48.518901 29941 raft_consensus.cc:729] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9437c62f856a412fb211229a40e7cd8d, State: Initialized, Role: FOLLOWER
I20260806 18:48:48.519150 29940 ts_tablet_manager.cc:1428] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:48:48.519330 29941 consensus_queue.cc:260] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.520306 29941 ts_tablet_manager.cc:1428] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:48:48.520687 29512 heartbeater.cc:503] Master 127.1.245.126:42115 was elected leader, sending a full tablet report...
I20260806 18:48:48.522475 29942 raft_consensus.cc:348] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.522696 29942 raft_consensus.cc:374] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:48.522753 29942 raft_consensus.cc:729] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feb3e441680943619a1ce237070a838c, State: Initialized, Role: FOLLOWER
I20260806 18:48:48.523204 29942 consensus_queue.cc:260] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.524289 29917 heartbeater.cc:503] Master 127.1.245.126:42115 was elected leader, sending a full tablet report...
I20260806 18:48:48.524616 29942 ts_tablet_manager.cc:1428] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260806 18:48:48.537575 29948 raft_consensus.cc:482] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:48.537781 29948 raft_consensus.cc:504] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.539072 29948 leader_election.cc:288] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 695b59b26b9c44a1bb5142c96364b5cc (127.1.245.66:45751), feb3e441680943619a1ce237070a838c (127.1.245.67:39435)
I20260806 18:48:48.543251 29515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.539260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:45751 (local address 127.1.245.65:40187)
0806 18:48:48.539403 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.539419 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:48.539512 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:48.540016 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:48.540020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:48.540044 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:48.540167 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.540176 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.541070 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.541074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.542001 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.542012 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.542814 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.542818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.542924 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.542950 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:48:48.543029 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 18:48:48.543097 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":36}
I20260806 18:48:48.543289 29934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.539613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:40187 (local address 127.1.245.66:45751)
0806 18:48:48.539761 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.539766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.539782 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:48:48.539836 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.539838 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.539894 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.539977 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:48.540317 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.540949 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.542140 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.542673 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.542781 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.542824 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.543055 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.543144 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:48:48.543366 29939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.539845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:48459 (local address 127.1.245.67:39435)
0806 18:48:48.540010 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.540014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.540028 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:48:48.540108 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.540110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.540155 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.540243 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:48.540597 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.541171 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.542387 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.542827 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.542950 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.543007 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.543174 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.543259 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260806 18:48:48.543419 29950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.539545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:39435 (local address 127.1.245.65:48459)
0806 18:48:48.539937 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.539951 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:48.540023 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:48.540311 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:48.540314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:48.540336 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:48.540470 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.540477 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.541315 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.541318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.542208 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:48.542219 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.542953 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.542955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:48.543047 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.543071 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:48:48.543144 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 18:48:48.543212 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":35,"thread_start_us":90,"threads_started":1}
I20260806 18:48:48.543994 29642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e24b30e3a342e5a5c74c88dfbee7d8" candidate_uuid: "9437c62f856a412fb211229a40e7cd8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "695b59b26b9c44a1bb5142c96364b5cc" is_pre_election: true
I20260806 18:48:48.544016 29839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e24b30e3a342e5a5c74c88dfbee7d8" candidate_uuid: "9437c62f856a412fb211229a40e7cd8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feb3e441680943619a1ce237070a838c" is_pre_election: true
I20260806 18:48:48.544288 29642 raft_consensus.cc:2454] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9437c62f856a412fb211229a40e7cd8d in term 0.
I20260806 18:48:48.544314 29839 raft_consensus.cc:2454] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9437c62f856a412fb211229a40e7cd8d in term 0.
I20260806 18:48:48.544751 29345 leader_election.cc:302] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695b59b26b9c44a1bb5142c96364b5cc, 9437c62f856a412fb211229a40e7cd8d; no voters:
I20260806 18:48:48.545006 29948 raft_consensus.cc:2781] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:48:48.545094 29948 raft_consensus.cc:482] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:48:48.545141 29948 raft_consensus.cc:3037] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:48.546177 29948 raft_consensus.cc:504] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.546569 29948 leader_election.cc:288] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [CANDIDATE]: Term 1 election: Requested vote from peers 695b59b26b9c44a1bb5142c96364b5cc (127.1.245.66:45751), feb3e441680943619a1ce237070a838c (127.1.245.67:39435)
I20260806 18:48:48.547004 29642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e24b30e3a342e5a5c74c88dfbee7d8" candidate_uuid: "9437c62f856a412fb211229a40e7cd8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "695b59b26b9c44a1bb5142c96364b5cc"
I20260806 18:48:48.547021 29839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "05e24b30e3a342e5a5c74c88dfbee7d8" candidate_uuid: "9437c62f856a412fb211229a40e7cd8d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feb3e441680943619a1ce237070a838c"
I20260806 18:48:48.547122 29642 raft_consensus.cc:3037] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:48.547127 29839 raft_consensus.cc:3037] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:48.548161 29642 raft_consensus.cc:2454] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9437c62f856a412fb211229a40e7cd8d in term 1.
I20260806 18:48:48.548166 29839 raft_consensus.cc:2454] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9437c62f856a412fb211229a40e7cd8d in term 1.
I20260806 18:48:48.548460 29345 leader_election.cc:302] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 695b59b26b9c44a1bb5142c96364b5cc, 9437c62f856a412fb211229a40e7cd8d; no voters:
I20260806 18:48:48.548609 29948 raft_consensus.cc:2781] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:48.548840 29948 raft_consensus.cc:686] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [term 1 LEADER]: Becoming Leader. State: Replica: 9437c62f856a412fb211229a40e7cd8d, State: Running, Role: LEADER
I20260806 18:48:48.549154 29948 consensus_queue.cc:237] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } }
I20260806 18:48:48.552736 29241 catalog_manager.cc:5495] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d reported cstate change: term changed from 0 to 1, leader changed from <none> to 9437c62f856a412fb211229a40e7cd8d (127.1.245.65). New cstate: current_term: 1 leader_uuid: "9437c62f856a412fb211229a40e7cd8d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9437c62f856a412fb211229a40e7cd8d" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 44661 } health_report { overall_health: HEALTHY } } }
W20260806 18:48:48.571523 29513 tablet.cc:2369] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:48.577325 29938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.573790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38788 (local address 127.1.245.65:44661)
0806 18:48:48.573942 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:48.573947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:48.573962 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:48:48.574037 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:48.574039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:48.574095 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:48.574189 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:48:48.574465 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.574979 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.575855 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:48.576078 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:48.576167 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:48.576473 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:48:48.576883 (+ 410us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:48:48.577028 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:48.577129 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 18:48:48.577165 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:48:48.583292 29839 raft_consensus.cc:1261] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 1 FOLLOWER]: Refusing update from remote peer 9437c62f856a412fb211229a40e7cd8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:48.583292 29642 raft_consensus.cc:1261] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 1 FOLLOWER]: Refusing update from remote peer 9437c62f856a412fb211229a40e7cd8d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:48.583936 29948 consensus_queue.cc:1035] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "695b59b26b9c44a1bb5142c96364b5cc" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 45751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:48.584131 29952 consensus_queue.cc:1035] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 9437c62f856a412fb211229a40e7cd8d [LEADER]: Connected to new peer: Peer: permanent_uuid: "feb3e441680943619a1ce237070a838c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 39435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:48.592845 29957 mvcc.cc:205] Tried to move back new op lower bound from 7315628558669987840 to 7315628558540627968. Current Snapshot: MvccSnapshot[applied={T|T < 7315628558669987840}]
I20260806 18:48:48.592492 29958 mvcc.cc:205] Tried to move back new op lower bound from 7315628558669987840 to 7315628558540627968. Current Snapshot: MvccSnapshot[applied={T|T < 7315628558669987840}]
I20260806 18:48:48.600567 29241 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:48.600711 29241 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:48.603017 29241 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f47056ddb1f347d5b91129dd3d9cd7ff: Sending DeleteTablet for 3 replicas of tablet 05e24b30e3a342e5a5c74c88dfbee7d8
I20260806 18:48:48.603869 29416 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e24b30e3a342e5a5c74c88dfbee7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:48 UTC) from {username='slave'} at 127.0.0.1:38776
I20260806 18:48:48.603946 29603 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e24b30e3a342e5a5c74c88dfbee7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:48 UTC) from {username='slave'} at 127.0.0.1:37492
I20260806 18:48:48.604346 29814 tablet_service.cc:1514] Processing DeleteTablet for tablet 05e24b30e3a342e5a5c74c88dfbee7d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:48 UTC) from {username='slave'} at 127.0.0.1:58138
I20260806 18:48:48.604393 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 29324
I20260806 18:48:48.611152 29973 tablet_replica.cc:330] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: stopping tablet replica
I20260806 18:48:48.611445 29973 raft_consensus.cc:2229] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:48.611730 29973 raft_consensus.cc:2258] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:48.612128 29976 tablet_replica.cc:330] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: stopping tablet replica
I20260806 18:48:48.612422 29976 raft_consensus.cc:2229] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:48.612721 29976 raft_consensus.cc:2258] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:48.613262 29973 ts_tablet_manager.cc:1905] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:48.614188 29976 ts_tablet_manager.cc:1905] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:48.615536 29973 ts_tablet_manager.cc:1918] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:48.615624 29973 log.cc:1196] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/05e24b30e3a342e5a5c74c88dfbee7d8
I20260806 18:48:48.615881 29973 ts_tablet_manager.cc:1939] T 05e24b30e3a342e5a5c74c88dfbee7d8 P 695b59b26b9c44a1bb5142c96364b5cc: Deleting consensus metadata
I20260806 18:48:48.616756 29976 ts_tablet_manager.cc:1918] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:48.616844 29976 log.cc:1196] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/05e24b30e3a342e5a5c74c88dfbee7d8
I20260806 18:48:48.616998 29219 catalog_manager.cc:4841] TS 695b59b26b9c44a1bb5142c96364b5cc (127.1.245.66:45751): tablet 05e24b30e3a342e5a5c74c88dfbee7d8 (table test-table [id=a22399dde4334f5a89a522a222859aa3]) successfully deleted
I20260806 18:48:48.617096 29976 ts_tablet_manager.cc:1939] T 05e24b30e3a342e5a5c74c88dfbee7d8 P feb3e441680943619a1ce237070a838c: Deleting consensus metadata
I20260806 18:48:48.618197 29219 catalog_manager.cc:4841] TS feb3e441680943619a1ce237070a838c (127.1.245.67:39435): tablet 05e24b30e3a342e5a5c74c88dfbee7d8 (table test-table [id=a22399dde4334f5a89a522a222859aa3]) successfully deleted
W20260806 18:48:48.618635 29225 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:44661 (error 108)
I20260806 18:48:48.618795 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 29516
I20260806 18:48:48.623255 29933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:48.622942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:44661 (local address 127.0.0.1:38798)
0806 18:48:48.623080 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:48.623163 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:44661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:48:48.623437 29225 catalog_manager.cc:4584] TS 9437c62f856a412fb211229a40e7cd8d (127.1.245.65:44661): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 05e24b30e3a342e5a5c74c88dfbee7d8: Network error: Client connection negotiation failed: client connection to 127.1.245.65:44661: connect: Connection refused (error 111)
I20260806 18:48:48.630625 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 29714
I20260806 18:48:48.640347 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 29195
2026-08-06T18:48:48Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1043 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:48 dist-test-slave-fgb3 krb5kdc[29980](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:48 dist-test-slave-fgb3 krb5kdc[29980](info): set up 2 sockets
Aug 06 18:48:48 dist-test-slave-fgb3 krb5kdc[29980](info): commencing operation
krb5kdc: starting...
W20260806 18:48:49.771071 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:48:49 dist-test-slave-fgb3 krb5kdc[29980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:48:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:48:49.898057 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:38705
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:41629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:38705
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:50.008584 30001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:50.008862 30001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:50.008905 30001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:50.012991 30001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:50.013053 30001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:50.013072 30001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:50.013087 30001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:50.013100 30001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:50.018369 30001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:41629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:38705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:38705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:50.019450 30001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:50.020532 30001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:50.028954 30012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:50.028939 30015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:50.028960 30013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:50.029194 30001 server_base.cc:1034] running on GCE node
I20260806 18:48:50.029913 30001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:50.031028 30001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:50.032202 30001 hybrid_clock.cc:648] HybridClock initialized: now 1786042130032186 us; error 29 us; skew 500 ppm
Aug 06 18:48:50 dist-test-slave-fgb3 krb5kdc[29980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:50.034997 30001 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:48:50.035929 30001 webserver.cc:460] Webserver started at http://127.1.245.126:45097/ using document root <none> and password file <none>
I20260806 18:48:50.036490 30001 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:50.036556 30001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:50.036756 30001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:50.038892 30001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "860d298de5cfbe2f2e706f690e26750d"
server_key_iv: "e05fa6f198649d248c4f155f87feae79"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.039460 30001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "860d298de5cfbe2f2e706f690e26750d"
server_key_iv: "e05fa6f198649d248c4f155f87feae79"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.043282 30001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:48:50.045792 30028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.047102 30001 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:48:50.047225 30001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "860d298de5cfbe2f2e706f690e26750d"
server_key_iv: "e05fa6f198649d248c4f155f87feae79"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.047338 30001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:50.073642 30001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:50.077005 30001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:50.077214 30001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:50.084439 30001 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:38705
I20260806 18:48:50.084481 30111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:38705 every 8 connection(s)
I20260806 18:48:50.085428 30001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:50.088220 30113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:50.093873 30113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0: Bootstrap starting.
I20260806 18:48:50.094923 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 30001
I20260806 18:48:50.095043 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:50.095289 2005 external_mini_cluster.cc:1428] Setting key ac2703a7cfe59405045a4543240c5f27
I20260806 18:48:50.096642 30113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:50.097378 30113 log.cc:824] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:50.099747 30113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0: No bootstrap required, opened a new log
I20260806 18:48:50.104053 30113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38705 } }
I20260806 18:48:50.104291 30113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:50.104349 30113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4484d5dedc6b4ee4b9a0a23435dbd5c0, State: Initialized, Role: FOLLOWER
Aug 06 18:48:50 dist-test-slave-fgb3 krb5kdc[29980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:50.104871 30113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38705 } }
I20260806 18:48:50.105044 30113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:50.105111 30113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:50.105209 30113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:50.106330 30113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38705 } }
I20260806 18:48:50.106689 30113 leader_election.cc:302] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4484d5dedc6b4ee4b9a0a23435dbd5c0; no voters:
I20260806 18:48:50.107033 30113 leader_election.cc:288] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:50.107118 30119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:50.107342 30119 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [term 1 LEADER]: Becoming Leader. State: Replica: 4484d5dedc6b4ee4b9a0a23435dbd5c0, State: Running, Role: LEADER
I20260806 18:48:50.107697 30119 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38705 } }
I20260806 18:48:50.108409 30113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:50.109417 30122 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38705 } } }
I20260806 18:48:50.109532 30122 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:50.109908 30123 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4484d5dedc6b4ee4b9a0a23435dbd5c0. Latest consensus state: current_term: 1 leader_uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4484d5dedc6b4ee4b9a0a23435dbd5c0" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38705 } } }
I20260806 18:48:50.110038 30123 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:50.110499 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.096403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37258 (local address 127.1.245.126:38705)
0806 18:48:50.096844 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:50.096861 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:50.096897 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:48:50.097544 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:50.097566 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:50.097975 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:50.098352 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:50.099286 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.100335 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.101535 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.102016 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.102136 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.105059 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:50.105065 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:50.105079 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:50.105113 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:50.106903 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.107428 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.107434 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.107440 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.107581 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.107906 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.107910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.107912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.108461 (+ 549us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:50.108641 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:50.109090 (+ 449us) server_negotiation.cc:299] Negotiation successful
0806 18:48:50.109445 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":122,"threads_started":1}
W20260806 18:48:50.113550 30147 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:48:50.113629 30147 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:48:50.113727 30133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:50.114553 30133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:50.120463 30133 catalog_manager.cc:1347] Generated new cluster ID: cb0a5969f4f34678aa9a3ac47448ca5f
I20260806 18:48:50.120523 30133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:50.136435 30133 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:50.137506 30133 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:50.143868 30133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0: Generated new TSK 0
I20260806 18:48:50.144696 30133 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.245.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
Entry for principal kudu/127.1.245.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab.
I20260806 18:48:50.180850 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38705
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:41629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:50.295126 30154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:50.295413 30154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:50.295473 30154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:50.299468 30154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:50.299528 30154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:50.299608 30154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:48:50.304908 30154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:41629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.65.keytab
--principal=kudu/127.1.245.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:50.305982 30154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:50.307044 30154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:50.317206 30167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:50.317206 30169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:50.317289 30166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:50.317394 30154 server_base.cc:1034] running on GCE node
I20260806 18:48:50.317932 30154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:50.318658 30154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:50.319828 30154 hybrid_clock.cc:648] HybridClock initialized: now 1786042130319805 us; error 32 us; skew 500 ppm
Aug 06 18:48:50 dist-test-slave-fgb3 krb5kdc[29980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:50.322636 30154 init.cc:377] Logged in from keytab as kudu/127.1.245.65@KRBTEST.COM (short username kudu)
I20260806 18:48:50.323493 30154 webserver.cc:460] Webserver started at http://127.1.245.65:38561/ using document root <none> and password file <none>
I20260806 18:48:50.324047 30154 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:50.324115 30154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:50.324331 30154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:50.326341 30154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "9af027d884f143e69df0d7648b441068"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "f4000b0af9bd0d4ccd5abcbc992063ab"
server_key_iv: "e4cfcf0b73cc85238617e5dddac677b3"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.326889 30154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "9af027d884f143e69df0d7648b441068"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "f4000b0af9bd0d4ccd5abcbc992063ab"
server_key_iv: "e4cfcf0b73cc85238617e5dddac677b3"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.330758 30154 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:48:50.333112 30176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.334375 30154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:48:50.334484 30154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "9af027d884f143e69df0d7648b441068"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "f4000b0af9bd0d4ccd5abcbc992063ab"
server_key_iv: "e4cfcf0b73cc85238617e5dddac677b3"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.334578 30154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:50.349677 30154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:50.352916 30154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:50.353123 30154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:50.353765 30154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:50.354671 30154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:50.354727 30154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.354785 30154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:50.354818 30154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.364009 30154 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:37785
I20260806 18:48:50.364068 30345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:37785 every 8 connection(s)
I20260806 18:48:50.364925 30154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:48:50.368461 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 30154
I20260806 18:48:50.368588 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:48:50.368861 2005 external_mini_cluster.cc:1428] Setting key de2a2120d3972766e7709696b30a4981
Aug 06 18:48:50 dist-test-slave-fgb3 krb5kdc[29980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.65@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.66@KRBTEST.COM; defaulting to no policy
I20260806 18:48:50.378844 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.366849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:43097 (local address 127.1.245.126:38705)
0806 18:48:50.367021 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:50.367025 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:50.367811 (+ 786us) server_negotiation.cc:407] Connection header received
0806 18:48:50.369183 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:50.369187 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:50.369255 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:50.369327 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:50.370675 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.371189 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.372329 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.372557 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.372646 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.375031 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:50.375034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:50.375036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:50.375061 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:50.376455 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.377073 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.377077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.377079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.377183 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.377561 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.377565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.377567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.377778 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:50.377876 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:50.378499 (+ 623us) server_negotiation.cc:299] Negotiation successful
0806 18:48:50.378643 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260806 18:48:50.379712 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.367188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38705 (local address 127.1.245.65:43097)
0806 18:48:50.367646 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:50.367688 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:50.368972 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:50.369512 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:50.369521 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:50.370006 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:50.370531 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.370546 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.371326 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.371329 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.372206 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.372216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.372700 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.372704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.372871 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.373413 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:50.373430 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:50.374866 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:50.376643 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:50.376650 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:50.376665 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:50.376956 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:50.377286 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:50.377290 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:50.377292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:50.377461 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:50.377886 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:50.377894 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:50.378186 (+ 292us) client_negotiation.cc:769] Sending connection context
0806 18:48:50.378443 (+ 257us) client_negotiation.cc:240] Negotiation successful
0806 18:48:50.378791 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
I20260806 18:48:50.381023 30348 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38705
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.66@KRBTEST.COM" created.
I20260806 18:48:50.381274 30348 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:50.381773 30348 heartbeater.cc:511] Master 127.1.245.126:38705 requested a full tablet report, sending...
I20260806 18:48:50.383278 30053 ts_manager.cc:194] Registered new tserver with Master: 9af027d884f143e69df0d7648b441068 (127.1.245.65:37785)
I20260806 18:48:50.384399 30053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.65@KRBTEST.COM'} at 127.1.245.65:43097
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
Entry for principal kudu/127.1.245.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab.
I20260806 18:48:50.396579 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38705
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:41629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:50.510146 30353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:50.510460 30353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:50.510510 30353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:50.514724 30353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:50.514788 30353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:50.514846 30353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:48:50.520185 30353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:41629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.66.keytab
--principal=kudu/127.1.245.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:50.521296 30353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:50.522387 30353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:50.532737 30360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:50.532769 30361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:50.532981 30353 server_base.cc:1034] running on GCE node
W20260806 18:48:50.533185 30364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:50.533723 30353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:50.534649 30353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:50.538394 30353 hybrid_clock.cc:648] HybridClock initialized: now 1786042130538362 us; error 41 us; skew 500 ppm
Aug 06 18:48:50 dist-test-slave-fgb3 krb5kdc[29980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:50.543974 30353 init.cc:377] Logged in from keytab as kudu/127.1.245.66@KRBTEST.COM (short username kudu)
I20260806 18:48:50.545238 30353 webserver.cc:460] Webserver started at http://127.1.245.66:45279/ using document root <none> and password file <none>
I20260806 18:48:50.546080 30353 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:50.546183 30353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:50.546471 30353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:50.548709 30353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "aa21c743159340faaba8b19bab937534"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "977bd7316b652c502bac3db031e1b67e"
server_key_iv: "7bb7fb330329678b8d19c81395093b20"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.549312 30353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "aa21c743159340faaba8b19bab937534"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "977bd7316b652c502bac3db031e1b67e"
server_key_iv: "7bb7fb330329678b8d19c81395093b20"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.553527 30353 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 18:48:50.556139 30377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.557440 30353 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:48:50.557559 30353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "aa21c743159340faaba8b19bab937534"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "977bd7316b652c502bac3db031e1b67e"
server_key_iv: "7bb7fb330329678b8d19c81395093b20"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.557662 30353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:50.576550 30353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:50.579663 30353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:50.579864 30353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:50.580483 30353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:50.581353 30353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:50.581403 30353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.581457 30353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:50.581486 30353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.589972 30353 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:41857
I20260806 18:48:50.590034 30535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:41857 every 8 connection(s)
I20260806 18:48:50.590925 30353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:48:50.595283 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 30353
I20260806 18:48:50.595433 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:48:50.595745 2005 external_mini_cluster.cc:1428] Setting key bd51fd1b414f067a0186179a1bcb9c54
Aug 06 18:48:50 dist-test-slave-fgb3 krb5kdc[29980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.66@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.245.67@KRBTEST.COM; defaulting to no policy
I20260806 18:48:50.605590 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.592946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:60845 (local address 127.1.245.126:38705)
0806 18:48:50.593098 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:50.593102 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:50.593957 (+ 855us) server_negotiation.cc:407] Connection header received
0806 18:48:50.595337 (+ 1380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:50.595341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:50.595404 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:50.595504 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:50.596790 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.597433 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.598673 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.598998 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.599087 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.601877 (+ 2790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:50.601881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:50.601883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:50.601915 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:50.603496 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.604030 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.604034 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.604035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.604116 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.604455 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.604458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.604460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.604656 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:50.604756 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:50.605286 (+ 530us) server_negotiation.cc:299] Negotiation successful
0806 18:48:50.605419 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260806 18:48:50.606539 30540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.593309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38705 (local address 127.1.245.66:60845)
0806 18:48:50.593779 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:50.593825 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:50.595092 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:50.595670 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:50.595677 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:50.596081 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:50.596657 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.596671 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.597561 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.597564 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.598558 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.598569 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.599153 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.599157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.599359 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.600002 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:50.600024 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:50.601683 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:50.603625 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:50.603631 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:50.603651 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:50.603927 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:50.604213 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:50.604216 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:50.604218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:50.604365 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:50.604761 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:50.604769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:50.605045 (+ 276us) client_negotiation.cc:769] Sending connection context
0806 18:48:50.605235 (+ 190us) client_negotiation.cc:240] Negotiation successful
0806 18:48:50.605600 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":118,"threads_started":1}
I20260806 18:48:50.607738 30536 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38705
I20260806 18:48:50.608000 30536 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.67@KRBTEST.COM" created.
I20260806 18:48:50.608709 30536 heartbeater.cc:511] Master 127.1.245.126:38705 requested a full tablet report, sending...
I20260806 18:48:50.609911 30053 ts_manager.cc:194] Registered new tserver with Master: aa21c743159340faaba8b19bab937534 (127.1.245.66:41857)
I20260806 18:48:50.610637 30053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.66@KRBTEST.COM'} at 127.1.245.66:60845
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
Entry for principal kudu/127.1.245.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab.
I20260806 18:48:50.625671 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38705
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:41629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:50.740296 30544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:50.740577 30544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:50.740638 30544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:50.744716 30544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:50.744812 30544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:50.744899 30544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:48:50.750367 30544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:41629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.67.keytab
--principal=kudu/127.1.245.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:38705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:50.751509 30544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:50.752553 30544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:50.762521 30552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:50.762518 30556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:50.762569 30551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:50.762939 30544 server_base.cc:1034] running on GCE node
I20260806 18:48:50.763447 30544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:50.764125 30544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:50.765288 30544 hybrid_clock.cc:648] HybridClock initialized: now 1786042130765266 us; error 34 us; skew 500 ppm
Aug 06 18:48:50 dist-test-slave-fgb3 krb5kdc[29980](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:50.768186 30544 init.cc:377] Logged in from keytab as kudu/127.1.245.67@KRBTEST.COM (short username kudu)
I20260806 18:48:50.769178 30544 webserver.cc:460] Webserver started at http://127.1.245.67:37477/ using document root <none> and password file <none>
I20260806 18:48:50.769750 30544 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:50.769814 30544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:50.770015 30544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:50.772226 30544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "06122cd13e06496c998c980227502b2a"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "17d129dd2ee1440ae44916ff6caf3dec"
server_key_iv: "96fc292add9b7a7fb151d9a7a5415688"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.772799 30544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "06122cd13e06496c998c980227502b2a"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "17d129dd2ee1440ae44916ff6caf3dec"
server_key_iv: "96fc292add9b7a7fb151d9a7a5415688"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.776683 30544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:48:50.779389 30570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.780623 30544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:48:50.780752 30544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "06122cd13e06496c998c980227502b2a"
format_stamp: "Formatted at 2026-08-06 18:48:50 on dist-test-slave-fgb3"
server_key: "17d129dd2ee1440ae44916ff6caf3dec"
server_key_iv: "96fc292add9b7a7fb151d9a7a5415688"
server_key_version: "encryptionkey@0"
I20260806 18:48:50.780858 30544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:50.793936 30544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:50.796702 30544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:50.796877 30544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:50.797465 30544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:50.798300 30544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:50.798369 30544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.798422 30544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:50.798452 30544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:50.807202 30544 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:44635
I20260806 18:48:50.807271 30732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:44635 every 8 connection(s)
I20260806 18:48:50.808104 30544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:48:50.812821 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 30544
I20260806 18:48:50.812959 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:48:50.813230 2005 external_mini_cluster.cc:1428] Setting key 3dfb03f704cb6e20ce633cd5468517c6
Aug 06 18:48:50 dist-test-slave-fgb3 krb5kdc[29980](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042130, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.67@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:48:50.821383 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.810057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:47011 (local address 127.1.245.126:38705)
0806 18:48:50.810193 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:50.810197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:50.811130 (+ 933us) server_negotiation.cc:407] Connection header received
0806 18:48:50.812424 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:50.812428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:50.812486 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:50.812555 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:50.813784 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.814407 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.815521 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.815739 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.815825 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.818248 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:50.818251 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:50.818253 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:50.818280 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:50.819451 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.819921 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.819925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.819927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.820007 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.820311 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.820315 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.820316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.820505 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:50.820592 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:50.821084 (+ 492us) server_negotiation.cc:299] Negotiation successful
0806 18:48:50.821205 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:48:50.822162 30738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.810510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38705 (local address 127.1.245.67:47011)
0806 18:48:50.810973 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:50.811017 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:50.812203 (+ 1186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:50.812736 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:50.812742 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:50.813136 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:50.813639 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.813654 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.814521 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.814524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.815408 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.815417 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.815883 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.815887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.816055 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.816598 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:48:50.816614 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:50.818092 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:50.819554 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:50.819560 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:50.819576 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:50.819822 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:50.820088 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:48:50.820091 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:48:50.820093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:48:50.820224 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:48:50.820603 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:50.820610 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:50.820856 (+ 246us) client_negotiation.cc:769] Sending connection context
0806 18:48:50.821023 (+ 167us) client_negotiation.cc:240] Negotiation successful
0806 18:48:50.821345 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260806 18:48:50.823225 30735 heartbeater.cc:347] Connected to a master server at 127.1.245.126:38705
I20260806 18:48:50.823451 30735 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:50.823917 30735 heartbeater.cc:511] Master 127.1.245.126:38705 requested a full tablet report, sending...
I20260806 18:48:50.824888 30053 ts_manager.cc:194] Registered new tserver with Master: 06122cd13e06496c998c980227502b2a (127.1.245.67:44635)
I20260806 18:48:50.825443 30053 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.245.67@KRBTEST.COM'} at 127.1.245.67:47011
I20260806 18:48:50.826813 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:48:50.835498 30116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.828838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37264 (local address 127.1.245.126:38705)
0806 18:48:50.828961 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:50.828965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:50.829123 (+ 158us) server_negotiation.cc:407] Connection header received
0806 18:48:50.829232 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:50.829236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:50.829275 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:50.829339 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:50.830004 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.830550 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.831407 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.831613 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.831696 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.832887 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:50.832890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:50.832892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:50.832916 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:50.833964 (+ 1048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.834348 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.834352 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.834354 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.834435 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:50.834671 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:50.834674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:50.834676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:50.834880 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:50.834997 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:50.835223 (+ 226us) server_negotiation.cc:299] Negotiation successful
0806 18:48:50.835332 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:48:50.838116 30053 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:48:50.841420 30053 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:48:50.856725 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.851176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50152 (local address 127.1.245.65:37785)
0806 18:48:50.851882 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:50.851898 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:50.851976 (+ 78us) server_negotiation.cc:407] Connection header received
0806 18:48:50.852066 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:50.852071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:50.852225 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:50.852680 (+ 455us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:50.852828 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.853560 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.855205 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.856080 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.856192 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.856344 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:50.856510 (+ 166us) server_negotiation.cc:299] Negotiation successful
0806 18:48:50.856590 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":598,"thread_start_us":93,"threads_started":1}
I20260806 18:48:50.856766 30752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.850901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:37785 (local address 127.0.0.1:50152)
0806 18:48:50.851418 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:50.851470 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:50.851611 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:50.852368 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:50.852371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:50.852395 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:50.852518 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.852525 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.853688 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.853691 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.855078 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.855090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.856198 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.856201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.856320 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.856363 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 18:48:50.856481 (+ 118us) client_negotiation.cc:240] Negotiation successful
0806 18:48:50.856608 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"spinlock_wait_cycles":12544,"thread_start_us":93,"threads_started":1}
I20260806 18:48:50.857345 30750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.850378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:44635 (local address 127.0.0.1:46660)
0806 18:48:50.851814 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:50.851826 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:50.851897 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:50.852340 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:50.852346 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:50.852369 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:50.852507 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.852513 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.853986 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.853990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.855116 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.855124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.856937 (+ 1813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.856940 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.857033 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.857057 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:48:50.857121 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 18:48:50.857196 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1380,"mutex_wait_us":600,"spinlock_wait_cycles":768,"thread_start_us":95,"threads_started":1}
I20260806 18:48:50.857576 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.850374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:41857 (local address 127.0.0.1:41578)
0806 18:48:50.851419 (+ 1045us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:50.851470 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:50.851612 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:50.852525 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:50.852528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:50.852544 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:50.852613 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.852619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.855239 (+ 2620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.855242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.855937 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:50.855944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.857170 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.857174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:50.857268 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.857292 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:48:50.857363 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 18:48:50.857436 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"spinlock_wait_cycles":7808,"thread_start_us":92,"threads_started":1}
I20260806 18:48:50.857748 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.850979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.1.245.67:44635)
0806 18:48:50.851868 (+ 889us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:50.851880 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:50.851903 (+ 23us) server_negotiation.cc:407] Connection header received
0806 18:48:50.852001 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:50.852005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:50.852186 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:50.852324 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:50.852610 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.853828 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.855216 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.856079 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.856193 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.856344 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:50.857521 (+ 1177us) server_negotiation.cc:299] Negotiation successful
0806 18:48:50.857610 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778,"thread_start_us":96,"threads_started":1}
I20260806 18:48:50.857810 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:50.850577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41578 (local address 127.1.245.66:41857)
0806 18:48:50.850937 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:50.850949 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:50.851567 (+ 618us) server_negotiation.cc:407] Connection header received
0806 18:48:50.852002 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:50.852009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:50.852207 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:50.852360 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:50.852726 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.853844 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.856059 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:50.857030 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:50.857137 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:50.857272 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:50.857493 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 18:48:50.857594 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260806 18:48:50.860471 30641 tablet_service.cc:1467] Processing CreateTablet for tablet 0f13668eb9904f588e03396ee6d7ce87 (DEFAULT_TABLE table=test-table [id=f3f69f034c684e63aa13165b60b4a90e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:50.860505 30249 tablet_service.cc:1467] Processing CreateTablet for tablet 0f13668eb9904f588e03396ee6d7ce87 (DEFAULT_TABLE table=test-table [id=f3f69f034c684e63aa13165b60b4a90e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:50.860659 30452 tablet_service.cc:1467] Processing CreateTablet for tablet 0f13668eb9904f588e03396ee6d7ce87 (DEFAULT_TABLE table=test-table [id=f3f69f034c684e63aa13165b60b4a90e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:50.861459 30641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f13668eb9904f588e03396ee6d7ce87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:50.861574 30249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f13668eb9904f588e03396ee6d7ce87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:50.861574 30452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f13668eb9904f588e03396ee6d7ce87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:50.868512 30758 tablet_bootstrap.cc:492] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Bootstrap starting.
I20260806 18:48:50.869114 30760 tablet_bootstrap.cc:492] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068: Bootstrap starting.
I20260806 18:48:50.870620 30758 tablet_bootstrap.cc:654] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:50.870826 30760 tablet_bootstrap.cc:654] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:50.871641 30758 log.cc:824] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:50.871757 30760 log.cc:824] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:50.872318 30759 tablet_bootstrap.cc:492] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Bootstrap starting.
I20260806 18:48:50.873512 30758 tablet_bootstrap.cc:492] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: No bootstrap required, opened a new log
I20260806 18:48:50.873708 30758 ts_tablet_manager.cc:1397] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:48:50.873739 30760 tablet_bootstrap.cc:492] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068: No bootstrap required, opened a new log
I20260806 18:48:50.873945 30760 ts_tablet_manager.cc:1397] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260806 18:48:50.875264 30759 tablet_bootstrap.cc:654] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:50.876127 30759 log.cc:824] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:50.876809 30758 raft_consensus.cc:348] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:50.877087 30758 raft_consensus.cc:374] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:50.877152 30758 raft_consensus.cc:729] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06122cd13e06496c998c980227502b2a, State: Initialized, Role: FOLLOWER
I20260806 18:48:50.877565 30758 consensus_queue.cc:260] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:50.877607 30760 raft_consensus.cc:348] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:50.877859 30760 raft_consensus.cc:374] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:50.877921 30760 raft_consensus.cc:729] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9af027d884f143e69df0d7648b441068, State: Initialized, Role: FOLLOWER
I20260806 18:48:50.878307 30735 heartbeater.cc:503] Master 127.1.245.126:38705 was elected leader, sending a full tablet report...
I20260806 18:48:50.878429 30759 tablet_bootstrap.cc:492] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: No bootstrap required, opened a new log
I20260806 18:48:50.878396 30760 consensus_queue.cc:260] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:50.878607 30759 ts_tablet_manager.cc:1397] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260806 18:48:50.878652 30758 ts_tablet_manager.cc:1428] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:48:50.879316 30760 ts_tablet_manager.cc:1428] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260806 18:48:50.879668 30348 heartbeater.cc:503] Master 127.1.245.126:38705 was elected leader, sending a full tablet report...
I20260806 18:48:50.882284 30759 raft_consensus.cc:348] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:50.882589 30759 raft_consensus.cc:374] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:50.882659 30759 raft_consensus.cc:729] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa21c743159340faaba8b19bab937534, State: Initialized, Role: FOLLOWER
I20260806 18:48:50.883100 30759 consensus_queue.cc:260] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:50.883695 30536 heartbeater.cc:503] Master 127.1.245.126:38705 was elected leader, sending a full tablet report...
I20260806 18:48:50.883973 30759 ts_tablet_manager.cc:1428] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Time spent starting tablet: real 0.005s user 0.001s sys 0.003s
I20260806 18:48:51.024765 30769 raft_consensus.cc:482] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:51.025007 30769 raft_consensus.cc:504] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:51.026207 30769 leader_election.cc:288] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 06122cd13e06496c998c980227502b2a (127.1.245.67:44635), 9af027d884f143e69df0d7648b441068 (127.1.245.65:37785)
I20260806 18:48:51.030112 30757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.026725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:57891 (local address 127.1.245.65:37785)
0806 18:48:51.026902 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:51.026907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:51.026924 (+ 17us) server_negotiation.cc:407] Connection header received
0806 18:48:51.027017 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:51.027020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:51.027072 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:51.027166 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:51.027476 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.028031 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.029178 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.029583 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.029694 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.029751 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:51.029909 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 18:48:51.029982 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260806 18:48:51.030110 30540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.026533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:37785 (local address 127.1.245.66:57891)
0806 18:48:51.026786 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:51.026802 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:51.026908 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:51.027200 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:51.027203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:51.027224 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:51.027348 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:51.027354 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.028140 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.028143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:51.029054 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:51.029064 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.029694 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.029698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:51.029800 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.029819 (+ 19us) client_negotiation.cc:769] Sending connection context
0806 18:48:51.029885 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:48:51.029965 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":31}
I20260806 18:48:51.030282 30771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.026533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:44635 (local address 127.1.245.66:60191)
0806 18:48:51.027026 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:51.027039 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:51.027127 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:51.027347 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:51.027350 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:51.027366 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:51.027492 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:51.027498 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.028259 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.028262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:51.029206 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:51.029216 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.029859 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.029862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:51.029973 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.029991 (+ 18us) client_negotiation.cc:769] Sending connection context
0806 18:48:51.030057 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 18:48:51.030124 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":74,"thread_start_us":143,"threads_started":1}
I20260806 18:48:51.030265 30754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.026641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:60191 (local address 127.1.245.67:44635)
0806 18:48:51.026787 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:51.026793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:51.027120 (+ 327us) server_negotiation.cc:407] Connection header received
0806 18:48:51.027214 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:51.027218 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:51.027260 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:51.027321 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:51.027601 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.028145 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.029331 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.029741 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.029851 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.029904 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:51.030086 (+ 182us) server_negotiation.cc:299] Negotiation successful
0806 18:48:51.030160 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260806 18:48:51.030946 30669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f13668eb9904f588e03396ee6d7ce87" candidate_uuid: "aa21c743159340faaba8b19bab937534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06122cd13e06496c998c980227502b2a" is_pre_election: true
I20260806 18:48:51.031038 30277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f13668eb9904f588e03396ee6d7ce87" candidate_uuid: "aa21c743159340faaba8b19bab937534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9af027d884f143e69df0d7648b441068" is_pre_election: true
I20260806 18:48:51.031303 30669 raft_consensus.cc:2454] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa21c743159340faaba8b19bab937534 in term 0.
I20260806 18:48:51.031302 30277 raft_consensus.cc:2454] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aa21c743159340faaba8b19bab937534 in term 0.
I20260806 18:48:51.031802 30380 leader_election.cc:302] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9af027d884f143e69df0d7648b441068, aa21c743159340faaba8b19bab937534; no voters:
I20260806 18:48:51.032076 30769 raft_consensus.cc:2781] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:48:51.032163 30769 raft_consensus.cc:482] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:48:51.032215 30769 raft_consensus.cc:3037] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:51.033304 30769 raft_consensus.cc:504] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:51.033717 30769 leader_election.cc:288] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [CANDIDATE]: Term 1 election: Requested vote from peers 06122cd13e06496c998c980227502b2a (127.1.245.67:44635), 9af027d884f143e69df0d7648b441068 (127.1.245.65:37785)
I20260806 18:48:51.034113 30669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f13668eb9904f588e03396ee6d7ce87" candidate_uuid: "aa21c743159340faaba8b19bab937534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "06122cd13e06496c998c980227502b2a"
I20260806 18:48:51.034125 30277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f13668eb9904f588e03396ee6d7ce87" candidate_uuid: "aa21c743159340faaba8b19bab937534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9af027d884f143e69df0d7648b441068"
I20260806 18:48:51.034242 30669 raft_consensus.cc:3037] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:51.034245 30277 raft_consensus.cc:3037] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:51.035368 30669 raft_consensus.cc:2454] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa21c743159340faaba8b19bab937534 in term 1.
I20260806 18:48:51.035368 30277 raft_consensus.cc:2454] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa21c743159340faaba8b19bab937534 in term 1.
I20260806 18:48:51.035684 30380 leader_election.cc:302] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9af027d884f143e69df0d7648b441068, aa21c743159340faaba8b19bab937534; no voters:
I20260806 18:48:51.035902 30769 raft_consensus.cc:2781] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:51.036098 30769 raft_consensus.cc:686] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 1 LEADER]: Becoming Leader. State: Replica: aa21c743159340faaba8b19bab937534, State: Running, Role: LEADER
I20260806 18:48:51.036453 30769 consensus_queue.cc:237] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } }
I20260806 18:48:51.039912 30051 catalog_manager.cc:5495] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 reported cstate change: term changed from 0 to 1, leader changed from <none> to aa21c743159340faaba8b19bab937534 (127.1.245.66). New cstate: current_term: 1 leader_uuid: "aa21c743159340faaba8b19bab937534" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa21c743159340faaba8b19bab937534" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 41857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 } health_report { overall_health: UNKNOWN } } }
W20260806 18:48:51.059111 30736 tablet.cc:2369] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:48:51.092147 30537 tablet.cc:2369] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:51.093827 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.090067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41582 (local address 127.1.245.66:41857)
0806 18:48:51.090215 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:51.090220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:51.090233 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:48:51.090338 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:51.090342 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:51.090407 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:51.090499 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:48:51.090741 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.091233 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.092105 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.092465 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.092567 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.092896 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:48:51.093404 (+ 508us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:48:51.093515 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:51.093602 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 18:48:51.093674 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:48:51.099872 30277 raft_consensus.cc:1261] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [term 1 FOLLOWER]: Refusing update from remote peer aa21c743159340faaba8b19bab937534: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:51.099872 30669 raft_consensus.cc:1261] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 1 FOLLOWER]: Refusing update from remote peer aa21c743159340faaba8b19bab937534: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:51.100541 30772 consensus_queue.cc:1035] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9af027d884f143e69df0d7648b441068" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 37785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:51.100706 30769 consensus_queue.cc:1035] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [LEADER]: Connected to new peer: Peer: permanent_uuid: "06122cd13e06496c998c980227502b2a" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 44635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:51.108695 30780 mvcc.cc:205] Tried to move back new op lower bound from 7315628568978198528 to 7315628568728039424. Current Snapshot: MvccSnapshot[applied={T|T < 7315628568978198528}]
I20260806 18:48:51.109088 30775 mvcc.cc:205] Tried to move back new op lower bound from 7315628568978198528 to 7315628568728039424. Current Snapshot: MvccSnapshot[applied={T|T < 7315628568978198528}]
I20260806 18:48:51.118283 30053 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:51.118501 30053 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:51.120987 30053 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4484d5dedc6b4ee4b9a0a23435dbd5c0: Sending DeleteTablet for 3 replicas of tablet 0f13668eb9904f588e03396ee6d7ce87
I20260806 18:48:51.121776 30641 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f13668eb9904f588e03396ee6d7ce87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:51 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:46660
I20260806 18:48:51.121774 30249 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f13668eb9904f588e03396ee6d7ce87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:51 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:50152
I20260806 18:48:51.122291 30795 tablet_replica.cc:330] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068: stopping tablet replica
I20260806 18:48:51.122598 30452 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f13668eb9904f588e03396ee6d7ce87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:51 UTC) from {username='kudu', principal='kudu/127.1.245.126@KRBTEST.COM'} at 127.0.0.1:41578
I20260806 18:48:51.122686 30795 raft_consensus.cc:2229] T 0f13668eb9904f588e03396ee6d7ce87 P 9af027d884f143e69df0d7648b441068 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:51.123049 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 30154
I20260806 18:48:51.123064 30796 tablet_replica.cc:330] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: stopping tablet replica
I20260806 18:48:51.125468 30796 raft_consensus.cc:2229] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:48:51.125888 30796 raft_consensus.cc:2258] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:51.129599 30796 ts_tablet_manager.cc:1905] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:51.132436 30796 ts_tablet_manager.cc:1918] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:51.132525 30796 log.cc:1196] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/0f13668eb9904f588e03396ee6d7ce87
I20260806 18:48:51.132848 30796 ts_tablet_manager.cc:1939] T 0f13668eb9904f588e03396ee6d7ce87 P aa21c743159340faaba8b19bab937534: Deleting consensus metadata
I20260806 18:48:51.134070 30033 catalog_manager.cc:4841] TS aa21c743159340faaba8b19bab937534 (127.1.245.66:41857): tablet 0f13668eb9904f588e03396ee6d7ce87 (table test-table [id=f3f69f034c684e63aa13165b60b4a90e]) successfully deleted
I20260806 18:48:51.140452 30794 tablet_replica.cc:330] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: stopping tablet replica
I20260806 18:48:51.140817 30794 raft_consensus.cc:2229] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:51.141157 30794 raft_consensus.cc:2258] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:51.142724 30794 ts_tablet_manager.cc:1905] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:51.145404 30794 ts_tablet_manager.cc:1918] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:51.145507 30794 log.cc:1196] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/0f13668eb9904f588e03396ee6d7ce87
I20260806 18:48:51.145853 30794 ts_tablet_manager.cc:1939] T 0f13668eb9904f588e03396ee6d7ce87 P 06122cd13e06496c998c980227502b2a: Deleting consensus metadata
W20260806 18:48:51.146379 30029 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:37785 (error 108)
I20260806 18:48:51.147219 30034 catalog_manager.cc:4841] TS 06122cd13e06496c998c980227502b2a (127.1.245.67:44635): tablet 0f13668eb9904f588e03396ee6d7ce87 (table test-table [id=f3f69f034c684e63aa13165b60b4a90e]) successfully deleted
I20260806 18:48:51.147478 30753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.147138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:37785 (local address 127.0.0.1:50164)
0806 18:48:51.147297 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:51.147381 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:37785: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:48:51.147575 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 30353
W20260806 18:48:51.147655 30029 catalog_manager.cc:4584] TS 9af027d884f143e69df0d7648b441068 (127.1.245.65:37785): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f13668eb9904f588e03396ee6d7ce87: Network error: Client connection negotiation failed: client connection to 127.1.245.65:37785: connect: Connection refused (error 111)
I20260806 18:48:51.157229 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 30544
I20260806 18:48:51.167881 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 30001
2026-08-06T18:48:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2532 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T18:48:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:51Z Disabled control of system clock
I20260806 18:48:51.207170 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:40709
--webserver_interface=127.1.245.126
--webserver_port=0
--builtin_ntp_servers=127.1.245.84:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:40709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:48:51.324666 30805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:51.325057 30805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:51.325114 30805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:51.330142 30805 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:51.330214 30805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:51.330228 30805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:51.330245 30805 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:51.330258 30805 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:51.335342 30805 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:40709
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:40709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30805
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:51.336448 30805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:51.337882 30805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:48:51.346930 30805 server_base.cc:1034] running on GCE node
W20260806 18:48:51.347227 30817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:51.347220 30812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:51.347225 30815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:51.347975 30805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:51.349056 30805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:51.350222 30805 hybrid_clock.cc:648] HybridClock initialized: now 1786042131350203 us; error 36 us; skew 500 ppm
I20260806 18:48:51.352161 30805 webserver.cc:460] Webserver started at http://127.1.245.126:37259/ using document root <none> and password file <none>
I20260806 18:48:51.352713 30805 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:51.352775 30805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:51.352948 30805 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:51.355062 30805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "7b065b1197754eb4b412a66ec9f76a95"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "d515b2e7082c282da5ee3d57b8ddfd99"
server_key_iv: "1cd475b38b95961dc4dace95774eab97"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.355583 30805 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "7b065b1197754eb4b412a66ec9f76a95"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "d515b2e7082c282da5ee3d57b8ddfd99"
server_key_iv: "1cd475b38b95961dc4dace95774eab97"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.359228 30805 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:48:51.361676 30825 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:51.363157 30805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:48:51.363281 30805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "7b065b1197754eb4b412a66ec9f76a95"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "d515b2e7082c282da5ee3d57b8ddfd99"
server_key_iv: "1cd475b38b95961dc4dace95774eab97"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.363399 30805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:51.399132 30805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:51.399917 30805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:51.400105 30805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:51.406994 30805 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:40709
I20260806 18:48:51.407012 30898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:40709 every 8 connection(s)
I20260806 18:48:51.408028 30805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:51.411201 30899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:51.416872 30899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95: Bootstrap starting.
I20260806 18:48:51.417958 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 30805
I20260806 18:48:51.418066 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:51.418350 2005 external_mini_cluster.cc:1428] Setting key ff3f98cd220602078fc4177d92f7d7b3
I20260806 18:48:51.419641 30899 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:51.420409 30899 log.cc:824] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:51.422590 30899 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95: No bootstrap required, opened a new log
I20260806 18:48:51.426772 30899 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b065b1197754eb4b412a66ec9f76a95" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40709 } }
I20260806 18:48:51.426999 30899 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:51.427057 30899 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b065b1197754eb4b412a66ec9f76a95, State: Initialized, Role: FOLLOWER
I20260806 18:48:51.427448 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.419483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.1.245.126:40709)
0806 18:48:51.420182 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:51.420197 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:51.420234 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:48:51.420928 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:51.420948 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:51.421284 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:51.421573 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:51.421911 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.422948 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.424117 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.424603 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.424719 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.425579 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:51.425584 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:51.425597 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:51.425979 (+ 382us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:51.426038 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:51.426059 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:51.426134 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:51.426284 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 18:48:51.426598 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":138,"threads_started":1}
I20260806 18:48:51.427464 30899 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b065b1197754eb4b412a66ec9f76a95" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40709 } }
I20260806 18:48:51.427687 30899 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:51.427757 30899 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:51.427845 30899 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:51.428905 30899 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b065b1197754eb4b412a66ec9f76a95" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40709 } }
I20260806 18:48:51.429247 30899 leader_election.cc:302] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b065b1197754eb4b412a66ec9f76a95; no voters:
I20260806 18:48:51.429551 30899 leader_election.cc:288] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:51.429701 30909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:51.429919 30909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [term 1 LEADER]: Becoming Leader. State: Replica: 7b065b1197754eb4b412a66ec9f76a95, State: Running, Role: LEADER
I20260806 18:48:51.430248 30909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b065b1197754eb4b412a66ec9f76a95" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40709 } }
I20260806 18:48:51.430830 30899 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:51.432024 30912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b065b1197754eb4b412a66ec9f76a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b065b1197754eb4b412a66ec9f76a95" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40709 } } }
I20260806 18:48:51.432251 30912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:51.432047 30911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b065b1197754eb4b412a66ec9f76a95. Latest consensus state: current_term: 1 leader_uuid: "7b065b1197754eb4b412a66ec9f76a95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b065b1197754eb4b412a66ec9f76a95" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 40709 } } }
I20260806 18:48:51.432524 30911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:51.432966 30920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:51.436158 30920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:51.443105 30920 catalog_manager.cc:1347] Generated new cluster ID: 489a672b84f64faaa3e33e0bd57466eb
I20260806 18:48:51.443200 30920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:51.457563 30920 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:51.458923 30920 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:51.469643 30920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95: Generated new TSK 0
I20260806 18:48:51.470548 30920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:48:51.476656 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.65:0
--local_ip_for_outbound_sockets=127.1.245.65
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40709
--builtin_ntp_servers=127.1.245.84:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 18:48:51.589599 30952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:51.589871 30952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:51.589910 30952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:51.593858 30952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:51.593917 30952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:51.593976 30952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.65
I20260806 18:48:51.598862 30952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.245.65
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30952
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.65
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:51.600033 30952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:51.601490 30952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:51.611189 30960 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:51.611198 30962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:51.611179 30968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:51.611493 30952 server_base.cc:1034] running on GCE node
I20260806 18:48:51.611994 30952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:51.612608 30952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:51.613790 30952 hybrid_clock.cc:648] HybridClock initialized: now 1786042131613747 us; error 53 us; skew 500 ppm
I20260806 18:48:51.619863 30952 webserver.cc:460] Webserver started at http://127.1.245.65:44245/ using document root <none> and password file <none>
I20260806 18:48:51.620471 30952 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:51.620539 30952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:51.620735 30952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:51.622890 30952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/data/instance:
uuid: "0747a5c58e4c43ddbb9ca2483c6597d5"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "f693d57792bd118543fb4516f88b36eb"
server_key_iv: "16f8a4634b9a1b412ed25237f6fb7f22"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.623466 30952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance:
uuid: "0747a5c58e4c43ddbb9ca2483c6597d5"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "f693d57792bd118543fb4516f88b36eb"
server_key_iv: "16f8a4634b9a1b412ed25237f6fb7f22"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.627206 30952 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:48:51.629644 30982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:51.631038 30952 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:48:51.631158 30952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/wal
uuid: "0747a5c58e4c43ddbb9ca2483c6597d5"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "f693d57792bd118543fb4516f88b36eb"
server_key_iv: "16f8a4634b9a1b412ed25237f6fb7f22"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.631266 30952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:51.649828 30952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:51.650626 30952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:51.650820 30952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:51.651432 30952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:51.652448 30952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:51.652500 30952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:51.652554 30952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:51.652583 30952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:51.661156 30952 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.65:35237
I20260806 18:48:51.661237 31147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.65:35237 every 8 connection(s)
I20260806 18:48:51.662104 30952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/data/info.pb
I20260806 18:48:51.664126 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 30952
I20260806 18:48:51.664260 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-0/wal/instance
I20260806 18:48:51.664541 2005 external_mini_cluster.cc:1428] Setting key dcb9ff5db8973baf69d16f3cd2a11cc1
I20260806 18:48:51.666569 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.66:0
--local_ip_for_outbound_sockets=127.1.245.66
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40709
--builtin_ntp_servers=127.1.245.84:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:48:51.672710 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.663997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.65:48365 (local address 127.1.245.126:40709)
0806 18:48:51.664165 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:51.664170 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:51.665371 (+ 1201us) server_negotiation.cc:407] Connection header received
0806 18:48:51.667009 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:51.667013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:51.667072 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:51.667161 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:51.668218 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.668770 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.670387 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.670738 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.670863 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.672146 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:51.672149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:51.672152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:51.672167 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:51.672187 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:51.672195 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:51.672256 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:51.672469 (+ 213us) server_negotiation.cc:299] Negotiation successful
0806 18:48:51.672557 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:48:51.674191 31152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.664473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40709 (local address 127.1.245.65:48365)
0806 18:48:51.665215 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:51.665269 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:51.666794 (+ 1525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:51.667358 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:51.667365 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:51.667449 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:51.668073 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:51.668094 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.668900 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.668904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:51.670230 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:51.670243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.670935 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.670941 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:51.671149 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.671805 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:51.671808 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:51.671870 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:51.671966 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:51.672272 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:51.672281 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:51.672288 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:48:51.672433 (+ 145us) client_negotiation.cc:240] Negotiation successful
0806 18:48:51.673071 (+ 638us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260806 18:48:51.675623 31148 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40709
I20260806 18:48:51.675895 31148 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:51.676465 31148 heartbeater.cc:511] Master 127.1.245.126:40709 requested a full tablet report, sending...
I20260806 18:48:51.678155 30846 ts_manager.cc:194] Registered new tserver with Master: 0747a5c58e4c43ddbb9ca2483c6597d5 (127.1.245.65:35237)
I20260806 18:48:51.679347 30846 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.65:48365
W20260806 18:48:51.784579 31155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:51.784873 31155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:51.784924 31155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:51.788923 31155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:51.788988 31155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:51.789063 31155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.66
I20260806 18:48:51.793910 31155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.245.66
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.66
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:51.795101 31155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:51.796541 31155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:51.806195 31164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:51.806200 31161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:51.806222 31169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:51.806253 31155 server_base.cc:1034] running on GCE node
I20260806 18:48:51.806895 31155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:51.807508 31155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:51.808671 31155 hybrid_clock.cc:648] HybridClock initialized: now 1786042131808637 us; error 53 us; skew 500 ppm
I20260806 18:48:51.810524 31155 webserver.cc:460] Webserver started at http://127.1.245.66:44459/ using document root <none> and password file <none>
I20260806 18:48:51.811091 31155 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:51.811156 31155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:51.811367 31155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:51.813387 31155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/data/instance:
uuid: "149d83b43b3b4dc59fb26cca42c38921"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "6ef6ba1a9f18f4f40826835c59570e13"
server_key_iv: "92dda82195c52877215862515ee875b2"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.813933 31155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance:
uuid: "149d83b43b3b4dc59fb26cca42c38921"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "6ef6ba1a9f18f4f40826835c59570e13"
server_key_iv: "92dda82195c52877215862515ee875b2"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.817554 31155 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:48:51.820094 31175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:51.821307 31155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:48:51.821417 31155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/wal
uuid: "149d83b43b3b4dc59fb26cca42c38921"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "6ef6ba1a9f18f4f40826835c59570e13"
server_key_iv: "92dda82195c52877215862515ee875b2"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.821511 31155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:51.831521 31155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:51.832235 31155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:51.832412 31155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:51.833005 31155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:51.834009 31155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:51.834059 31155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:51.834113 31155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:51.834141 31155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:51.842696 31155 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.66:46715
I20260806 18:48:51.842741 31344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.66:46715 every 8 connection(s)
I20260806 18:48:51.843601 31155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/data/info.pb
I20260806 18:48:51.843986 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 31155
I20260806 18:48:51.844122 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/wal/instance
I20260806 18:48:51.844447 2005 external_mini_cluster.cc:1428] Setting key 44dc9030b532dede220ca976737d2439
I20260806 18:48:51.846586 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.245.67:0
--local_ip_for_outbound_sockets=127.1.245.67
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40709
--builtin_ntp_servers=127.1.245.84:37929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 18:48:51.853304 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.845334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.66:59473 (local address 127.1.245.126:40709)
0806 18:48:51.845477 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:51.845482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:51.846789 (+ 1307us) server_negotiation.cc:407] Connection header received
0806 18:48:51.848590 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:51.848593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:51.848652 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:51.848723 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:51.849618 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.850186 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.851406 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.851631 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.851723 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.852839 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:51.852842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:51.852844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:51.852857 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:51.852876 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:51.852883 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:51.852942 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:51.853081 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 18:48:51.853154 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 18:48:51.854156 31348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:51.845819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40709 (local address 127.1.245.66:59473)
0806 18:48:51.846629 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:51.846688 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:51.848387 (+ 1699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:51.848899 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:51.848906 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:51.848992 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:51.849479 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:51.849492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.850341 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.850344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:51.851284 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:51.851295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:51.851780 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:51.851784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:51.851941 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:51.852583 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:51.852584 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:51.852626 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:51.852693 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:51.852952 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:51.852961 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:51.852967 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 18:48:51.853074 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:48:51.853371 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":118,"threads_started":1}
I20260806 18:48:51.855458 31345 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40709
I20260806 18:48:51.855695 31345 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:51.856269 31345 heartbeater.cc:511] Master 127.1.245.126:40709 requested a full tablet report, sending...
I20260806 18:48:51.857371 30846 ts_manager.cc:194] Registered new tserver with Master: 149d83b43b3b4dc59fb26cca42c38921 (127.1.245.66:46715)
I20260806 18:48:51.857928 30846 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.66:59473
W20260806 18:48:51.958626 31349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:51.958871 31349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:51.958925 31349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:51.962841 31349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:51.962893 31349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:51.962965 31349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.245.67
I20260806 18:48:51.967593 31349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:37929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.245.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.245.67
--webserver_port=0
--tserver_master_addrs=127.1.245.126:40709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.245.67
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:51.968605 31349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:51.970074 31349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:51.979249 31358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:51.979329 31362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:51.979533 31360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:51.979530 31349 server_base.cc:1034] running on GCE node
I20260806 18:48:51.979972 31349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:51.980587 31349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:51.981736 31349 hybrid_clock.cc:648] HybridClock initialized: now 1786042131981717 us; error 33 us; skew 500 ppm
I20260806 18:48:51.983441 31349 webserver.cc:460] Webserver started at http://127.1.245.67:46291/ using document root <none> and password file <none>
I20260806 18:48:51.983966 31349 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:51.984031 31349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:51.984210 31349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:51.986187 31349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/data/instance:
uuid: "ad5363399a964a8b98d16bc2763b192c"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "a8c1f751883926c80717bdc7ef812ffa"
server_key_iv: "8be0cd9a91fc0bb5261fe9083bb32d68"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.986752 31349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance:
uuid: "ad5363399a964a8b98d16bc2763b192c"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "a8c1f751883926c80717bdc7ef812ffa"
server_key_iv: "8be0cd9a91fc0bb5261fe9083bb32d68"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.990078 31349 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:48:51.992443 31378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:51.993742 31349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260806 18:48:51.993876 31349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/wal
uuid: "ad5363399a964a8b98d16bc2763b192c"
format_stamp: "Formatted at 2026-08-06 18:48:51 on dist-test-slave-fgb3"
server_key: "a8c1f751883926c80717bdc7ef812ffa"
server_key_iv: "8be0cd9a91fc0bb5261fe9083bb32d68"
server_key_version: "encryptionkey@0"
I20260806 18:48:51.993979 31349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:52.010020 31349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:52.010725 31349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:52.010902 31349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:52.011472 31349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 18:48:52.012419 31349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 18:48:52.012467 31349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:52.012522 31349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 18:48:52.012547 31349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:52.021304 31349 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.67:41123
I20260806 18:48:52.021396 31542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.67:41123 every 8 connection(s)
I20260806 18:48:52.022351 31349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/data/info.pb
I20260806 18:48:52.023501 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 31349
I20260806 18:48:52.023618 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/wal/instance
I20260806 18:48:52.023941 2005 external_mini_cluster.cc:1428] Setting key 82ebdd7ba2130ce22d3d97edc5ab05d0
I20260806 18:48:52.032352 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.024587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:44169 (local address 127.1.245.126:40709)
0806 18:48:52.024764 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:52.024769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:52.026206 (+ 1437us) server_negotiation.cc:407] Connection header received
0806 18:48:52.027651 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:52.027654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:52.027702 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:52.027776 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:52.028667 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.029217 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.030405 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.030638 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.030722 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.031879 (+ 1157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:52.031882 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:52.031884 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:52.031897 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:52.031917 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:52.031925 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:52.031992 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:52.032142 (+ 150us) server_negotiation.cc:299] Negotiation successful
0806 18:48:52.032206 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:48:52.033284 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.025236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:40709 (local address 127.1.245.67:44169)
0806 18:48:52.026063 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:52.026118 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:52.027464 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:52.027921 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:52.027927 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:52.028016 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:48:52.028517 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.028530 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.029339 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.029342 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.030260 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.030273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.030783 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.030786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.030950 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.031625 (+ 675us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 18:48:52.031627 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:48:52.031669 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 18:48:52.031745 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:48:52.032009 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:48:52.032017 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:48:52.032024 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 18:48:52.032117 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 18:48:52.032350 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":140,"threads_started":1}
I20260806 18:48:52.034377 31543 heartbeater.cc:347] Connected to a master server at 127.1.245.126:40709
I20260806 18:48:52.034608 31543 heartbeater.cc:464] Registering TS with master...
I20260806 18:48:52.035176 31543 heartbeater.cc:511] Master 127.1.245.126:40709 requested a full tablet report, sending...
I20260806 18:48:52.036347 30846 ts_manager.cc:194] Registered new tserver with Master: ad5363399a964a8b98d16bc2763b192c (127.1.245.67:41123)
I20260806 18:48:52.036973 30846 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.245.67:44169
I20260806 18:48:52.038168 2005 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 18:48:52.043852 30904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.040486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47416 (local address 127.1.245.126:40709)
0806 18:48:52.040607 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:52.040611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:52.040712 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:48:52.040824 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:52.040826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:52.040864 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:52.040928 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:52.041183 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.041682 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.042555 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.042775 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.042849 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.043540 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:52.043543 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:52.043545 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:52.043557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:52.043575 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:52.043582 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:52.043645 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:52.043698 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 18:48:52.043745 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 18:48:52.046795 30846 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 18:48:52.049724 30846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 18:48:52.064261 31562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.058234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:35237 (local address 127.0.0.1:46954)
0806 18:48:52.059105 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:52.059158 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:52.059289 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:52.060195 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:52.060198 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:52.060211 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:52.060429 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.060438 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.061441 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.061444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.062677 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.062688 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.063661 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.063664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.063786 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.063841 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 18:48:52.063961 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 18:48:52.064102 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":44,"thread_start_us":103,"threads_started":1}
I20260806 18:48:52.064293 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.058579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46954 (local address 127.1.245.65:35237)
0806 18:48:52.059741 (+ 1162us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:52.059752 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:52.059777 (+ 25us) server_negotiation.cc:407] Connection header received
0806 18:48:52.059853 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:52.059859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:52.060042 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:52.060236 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:52.060554 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.061330 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.062821 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.063509 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.063650 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.063862 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:52.063996 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 18:48:52.064100 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1070,"thread_start_us":153,"threads_started":1}
I20260806 18:48:52.064782 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.058943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:46715 (local address 127.0.0.1:51324)
0806 18:48:52.059268 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:52.059280 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:52.059452 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:52.060162 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:52.060168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:52.060193 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:52.060487 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.060496 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.061910 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.061913 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.063014 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.063024 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.064260 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.064264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.064492 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.064518 (+ 26us) client_negotiation.cc:769] Sending connection context
0806 18:48:52.064583 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 18:48:52.064649 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"mutex_wait_us":114,"spinlock_wait_cycles":13056,"thread_start_us":95,"threads_started":1}
I20260806 18:48:52.065088 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.059082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.67:41123 (local address 127.0.0.1:58350)
0806 18:48:52.059322 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:52.059344 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:52.059451 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:52.060313 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:52.060315 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:52.060327 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:52.060476 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.060483 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.062402 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.062406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.063458 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.063469 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.064695 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.064698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.064798 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.064822 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 18:48:52.064890 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 18:48:52.064958 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":63,"spinlock_wait_cycles":10880,"thread_start_us":94,"threads_started":1}
I20260806 18:48:52.065279 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.058574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51324 (local address 127.1.245.66:46715)
0806 18:48:52.059069 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:52.059082 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:52.059370 (+ 288us) server_negotiation.cc:407] Connection header received
0806 18:48:52.059620 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:52.059627 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:52.059856 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:52.060002 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:52.060593 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.061778 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.063152 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.063935 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.064027 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.064198 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:52.065034 (+ 836us) server_negotiation.cc:299] Negotiation successful
0806 18:48:52.065123 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":160,"threads_started":1}
I20260806 18:48:52.065593 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.059595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.1.245.67:41123)
0806 18:48:52.059920 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:52.059929 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:52.059949 (+ 20us) server_negotiation.cc:407] Connection header received
0806 18:48:52.060014 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:52.060018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:52.060174 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:52.060305 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:52.060594 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.061778 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.063597 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.064355 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.064458 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.064632 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:52.065325 (+ 693us) server_negotiation.cc:299] Negotiation successful
0806 18:48:52.065437 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230,"thread_start_us":103,"threads_started":1}
I20260806 18:48:52.067827 31055 tablet_service.cc:1467] Processing CreateTablet for tablet 1b454a4606744136aba88d130d433987 (DEFAULT_TABLE table=test-table [id=0085a8083b5e4846a9d3d2ea2812a6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:52.068289 31448 tablet_service.cc:1467] Processing CreateTablet for tablet 1b454a4606744136aba88d130d433987 (DEFAULT_TABLE table=test-table [id=0085a8083b5e4846a9d3d2ea2812a6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:52.068861 31055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b454a4606744136aba88d130d433987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:52.069360 31448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b454a4606744136aba88d130d433987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:52.069130 31244 tablet_service.cc:1467] Processing CreateTablet for tablet 1b454a4606744136aba88d130d433987 (DEFAULT_TABLE table=test-table [id=0085a8083b5e4846a9d3d2ea2812a6c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 18:48:52.070106 31244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1b454a4606744136aba88d130d433987. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:52.075961 31574 tablet_bootstrap.cc:492] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5: Bootstrap starting.
I20260806 18:48:52.076500 31575 tablet_bootstrap.cc:492] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Bootstrap starting.
I20260806 18:48:52.077288 31576 tablet_bootstrap.cc:492] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Bootstrap starting.
I20260806 18:48:52.078430 31574 tablet_bootstrap.cc:654] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:52.079020 31576 tablet_bootstrap.cc:654] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:52.079018 31575 tablet_bootstrap.cc:654] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:52.079334 31574 log.cc:824] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:52.079711 31576 log.cc:824] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:52.079999 31575 log.cc:824] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:52.081467 31574 tablet_bootstrap.cc:492] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5: No bootstrap required, opened a new log
I20260806 18:48:52.081599 31576 tablet_bootstrap.cc:492] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: No bootstrap required, opened a new log
I20260806 18:48:52.081679 31574 ts_tablet_manager.cc:1397] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260806 18:48:52.081789 31576 ts_tablet_manager.cc:1397] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260806 18:48:52.081794 31575 tablet_bootstrap.cc:492] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: No bootstrap required, opened a new log
I20260806 18:48:52.081955 31575 ts_tablet_manager.cc:1397] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260806 18:48:52.084992 31576 raft_consensus.cc:348] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.085230 31576 raft_consensus.cc:374] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:52.085285 31576 raft_consensus.cc:729] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 149d83b43b3b4dc59fb26cca42c38921, State: Initialized, Role: FOLLOWER
I20260806 18:48:52.085726 31576 consensus_queue.cc:260] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.085944 31574 raft_consensus.cc:348] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.086210 31574 raft_consensus.cc:374] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:52.086272 31574 raft_consensus.cc:729] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0747a5c58e4c43ddbb9ca2483c6597d5, State: Initialized, Role: FOLLOWER
I20260806 18:48:52.086668 31574 consensus_queue.cc:260] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.086815 31576 ts_tablet_manager.cc:1428] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260806 18:48:52.086872 31575 raft_consensus.cc:348] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.087105 31575 raft_consensus.cc:374] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:52.087173 31575 raft_consensus.cc:729] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad5363399a964a8b98d16bc2763b192c, State: Initialized, Role: FOLLOWER
I20260806 18:48:52.087221 31345 heartbeater.cc:503] Master 127.1.245.126:40709 was elected leader, sending a full tablet report...
I20260806 18:48:52.087457 31148 heartbeater.cc:503] Master 127.1.245.126:40709 was elected leader, sending a full tablet report...
I20260806 18:48:52.087621 31574 ts_tablet_manager.cc:1428] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260806 18:48:52.087625 31575 consensus_queue.cc:260] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.088269 31543 heartbeater.cc:503] Master 127.1.245.126:40709 was elected leader, sending a full tablet report...
I20260806 18:48:52.089475 31575 ts_tablet_manager.cc:1428] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
W20260806 18:48:52.094684 31346 tablet.cc:2369] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 18:48:52.163352 31151 tablet.cc:2369] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:52.214443 31584 raft_consensus.cc:482] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 18:48:52.214718 31584 raft_consensus.cc:504] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.216221 31584 leader_election.cc:288] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0747a5c58e4c43ddbb9ca2483c6597d5 (127.1.245.65:35237), 149d83b43b3b4dc59fb26cca42c38921 (127.1.245.66:46715)
I20260806 18:48:52.220033 31564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.216635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:58573 (local address 127.1.245.65:35237)
0806 18:48:52.216977 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:52.216981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:52.216996 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:48:52.217045 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:52.217048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:52.217092 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:52.217174 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:52.217493 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.218014 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.219060 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.219437 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.219544 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.219664 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:52.219839 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 18:48:52.219914 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260806 18:48:52.220033 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.216501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:35237 (local address 127.1.245.67:58573)
0806 18:48:52.216674 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:52.216690 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:52.216788 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:52.217206 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:52.217209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:52.217268 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:52.217372 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.217378 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.218116 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.218119 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.218932 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.218942 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.219544 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.219548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.219718 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.219740 (+ 22us) client_negotiation.cc:769] Sending connection context
0806 18:48:52.219812 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:48:52.219887 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260806 18:48:52.220448 31585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.216522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.66:46715 (local address 127.1.245.67:37539)
0806 18:48:52.217188 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:52.217200 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:48:52.217330 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:48:52.217557 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:48:52.217560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:48:52.217575 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 18:48:52.217700 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.217705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.218437 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.218440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.219213 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:48:52.219224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.219975 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.219979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:48:52.220079 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.220102 (+ 23us) client_negotiation.cc:769] Sending connection context
0806 18:48:52.220215 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 18:48:52.220286 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"mutex_wait_us":164,"spinlock_wait_cycles":768,"thread_start_us":283,"threads_started":1}
I20260806 18:48:52.220463 31565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.216593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.245.67:37539 (local address 127.1.245.66:46715)
0806 18:48:52.216948 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:52.216953 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:52.217292 (+ 339us) server_negotiation.cc:407] Connection header received
0806 18:48:52.217429 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:52.217433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:52.217477 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:52.217539 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 18:48:52.217805 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.218336 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.219330 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.219832 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.219964 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.220012 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:52.220262 (+ 250us) server_negotiation.cc:299] Negotiation successful
0806 18:48:52.220347 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260806 18:48:52.220829 31088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b454a4606744136aba88d130d433987" candidate_uuid: "ad5363399a964a8b98d16bc2763b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" is_pre_election: true
I20260806 18:48:52.221179 31088 raft_consensus.cc:2454] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5363399a964a8b98d16bc2763b192c in term 0.
I20260806 18:48:52.221170 31275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b454a4606744136aba88d130d433987" candidate_uuid: "ad5363399a964a8b98d16bc2763b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "149d83b43b3b4dc59fb26cca42c38921" is_pre_election: true
I20260806 18:48:52.221431 31275 raft_consensus.cc:2454] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad5363399a964a8b98d16bc2763b192c in term 0.
I20260806 18:48:52.221684 31380 leader_election.cc:302] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0747a5c58e4c43ddbb9ca2483c6597d5, ad5363399a964a8b98d16bc2763b192c; no voters:
I20260806 18:48:52.221992 31584 raft_consensus.cc:2781] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 18:48:52.222072 31584 raft_consensus.cc:482] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 18:48:52.222101 31584 raft_consensus.cc:3037] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:52.223170 31584 raft_consensus.cc:504] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.223513 31584 leader_election.cc:288] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [CANDIDATE]: Term 1 election: Requested vote from peers 0747a5c58e4c43ddbb9ca2483c6597d5 (127.1.245.65:35237), 149d83b43b3b4dc59fb26cca42c38921 (127.1.245.66:46715)
I20260806 18:48:52.223886 31275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b454a4606744136aba88d130d433987" candidate_uuid: "ad5363399a964a8b98d16bc2763b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "149d83b43b3b4dc59fb26cca42c38921"
I20260806 18:48:52.223898 31088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1b454a4606744136aba88d130d433987" candidate_uuid: "ad5363399a964a8b98d16bc2763b192c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5"
I20260806 18:48:52.224001 31275 raft_consensus.cc:3037] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:52.224009 31088 raft_consensus.cc:3037] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:52.225090 31088 raft_consensus.cc:2454] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5363399a964a8b98d16bc2763b192c in term 1.
I20260806 18:48:52.225127 31275 raft_consensus.cc:2454] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad5363399a964a8b98d16bc2763b192c in term 1.
I20260806 18:48:52.225395 31380 leader_election.cc:302] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0747a5c58e4c43ddbb9ca2483c6597d5, ad5363399a964a8b98d16bc2763b192c; no voters:
I20260806 18:48:52.225560 31584 raft_consensus.cc:2781] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:52.225792 31584 raft_consensus.cc:686] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 1 LEADER]: Becoming Leader. State: Replica: ad5363399a964a8b98d16bc2763b192c, State: Running, Role: LEADER
I20260806 18:48:52.226169 31584 consensus_queue.cc:237] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } }
I20260806 18:48:52.229346 30846 catalog_manager.cc:5495] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c reported cstate change: term changed from 0 to 1, leader changed from <none> to ad5363399a964a8b98d16bc2763b192c (127.1.245.67). New cstate: current_term: 1 leader_uuid: "ad5363399a964a8b98d16bc2763b192c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ad5363399a964a8b98d16bc2763b192c" member_type: VOTER last_known_addr { host: "127.1.245.67" port: 41123 } health_report { overall_health: HEALTHY } } }
I20260806 18:48:52.261869 31570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.258022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58356 (local address 127.1.245.67:41123)
0806 18:48:52.258160 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:52.258163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:52.258184 (+ 21us) server_negotiation.cc:407] Connection header received
0806 18:48:52.258272 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:52.258275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:52.258360 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:52.258449 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:48:52.258753 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.259348 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.260312 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:52.260535 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:52.260626 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:52.260894 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:48:52.261359 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:48:52.261455 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:52.261617 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 18:48:52.261678 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:48:52.268404 31275 raft_consensus.cc:1261] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 1 FOLLOWER]: Refusing update from remote peer ad5363399a964a8b98d16bc2763b192c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:52.268404 31088 raft_consensus.cc:1261] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5 [term 1 FOLLOWER]: Refusing update from remote peer ad5363399a964a8b98d16bc2763b192c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:48:52.269305 31588 consensus_queue.cc:1035] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "0747a5c58e4c43ddbb9ca2483c6597d5" member_type: VOTER last_known_addr { host: "127.1.245.65" port: 35237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:48:52.269519 31584 consensus_queue.cc:1035] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [LEADER]: Connected to new peer: Peer: permanent_uuid: "149d83b43b3b4dc59fb26cca42c38921" member_type: VOTER last_known_addr { host: "127.1.245.66" port: 46715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 18:48:52.273350 31545 tablet.cc:2369] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:48:52.276842 31593 mvcc.cc:205] Tried to move back new op lower bound from 7315628573763207168 to 7315628573601165312. Current Snapshot: MvccSnapshot[applied={T|T < 7315628573763207168}]
I20260806 18:48:52.278580 31597 mvcc.cc:205] Tried to move back new op lower bound from 7315628573763207168 to 7315628573601165312. Current Snapshot: MvccSnapshot[applied={T|T < 7315628573763207168}]
I20260806 18:48:52.278780 31599 mvcc.cc:205] Tried to move back new op lower bound from 7315628573763207168 to 7315628573601165312. Current Snapshot: MvccSnapshot[applied={T|T < 7315628573763207168}]
I20260806 18:48:52.287077 30844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:52.287299 30844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 18:48:52.290153 30844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b065b1197754eb4b412a66ec9f76a95: Sending DeleteTablet for 3 replicas of tablet 1b454a4606744136aba88d130d433987
I20260806 18:48:52.291235 31244 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b454a4606744136aba88d130d433987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:52 UTC) from {username='slave'} at 127.0.0.1:51324
I20260806 18:48:52.291155 31055 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b454a4606744136aba88d130d433987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:52 UTC) from {username='slave'} at 127.0.0.1:46954
I20260806 18:48:52.291344 31448 tablet_service.cc:1514] Processing DeleteTablet for tablet 1b454a4606744136aba88d130d433987 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 18:48:52 UTC) from {username='slave'} at 127.0.0.1:58350
I20260806 18:48:52.291772 31606 tablet_replica.cc:330] T 1b454a4606744136aba88d130d433987 P 0747a5c58e4c43ddbb9ca2483c6597d5: stopping tablet replica
I20260806 18:48:52.291975 31608 tablet_replica.cc:330] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: stopping tablet replica
I20260806 18:48:52.292055 31607 tablet_replica.cc:330] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: stopping tablet replica
I20260806 18:48:52.292410 31608 raft_consensus.cc:2229] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 1 LEADER]: Raft consensus shutting down.
I20260806 18:48:52.292418 31607 raft_consensus.cc:2229] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 18:48:52.292577 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 30952
I20260806 18:48:52.292757 31607 raft_consensus.cc:2258] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:52.292969 31608 raft_consensus.cc:2258] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 18:48:52.294831 31607 ts_tablet_manager.cc:1905] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:52.296561 31608 ts_tablet_manager.cc:1905] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 18:48:52.297891 31607 ts_tablet_manager.cc:1918] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:52.298017 31607 log.cc:1196] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-1/wal/wals/1b454a4606744136aba88d130d433987
I20260806 18:48:52.298388 31607 ts_tablet_manager.cc:1939] T 1b454a4606744136aba88d130d433987 P 149d83b43b3b4dc59fb26cca42c38921: Deleting consensus metadata
I20260806 18:48:52.299803 31608 ts_tablet_manager.cc:1918] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 18:48:52.299969 31608 log.cc:1196] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Deleting WAL directory at /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786041874296929-2005-0/minicluster-data/ts-2/wal/wals/1b454a4606744136aba88d130d433987
I20260806 18:48:52.299933 30826 catalog_manager.cc:4841] TS 149d83b43b3b4dc59fb26cca42c38921 (127.1.245.66:46715): tablet 1b454a4606744136aba88d130d433987 (table test-table [id=0085a8083b5e4846a9d3d2ea2812a6c2]) successfully deleted
I20260806 18:48:52.300334 31608 ts_tablet_manager.cc:1939] T 1b454a4606744136aba88d130d433987 P ad5363399a964a8b98d16bc2763b192c: Deleting consensus metadata
I20260806 18:48:52.302771 30828 catalog_manager.cc:4841] TS ad5363399a964a8b98d16bc2763b192c (127.1.245.67:41123): tablet 1b454a4606744136aba88d130d433987 (table test-table [id=0085a8083b5e4846a9d3d2ea2812a6c2]) successfully deleted
W20260806 18:48:52.308780 30827 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.245.65:35237 (error 108)
I20260806 18:48:52.308877 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 31155
I20260806 18:48:52.314996 31568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:52.314630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.65:35237 (local address 127.0.0.1:46960)
0806 18:48:52.314787 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 18:48:52.314888 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.65:35237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 18:48:52.315332 30827 catalog_manager.cc:4584] TS 0747a5c58e4c43ddbb9ca2483c6597d5 (127.1.245.65:35237): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1b454a4606744136aba88d130d433987: Network error: Client connection negotiation failed: client connection to 127.1.245.65:35237: connect: Connection refused (error 111)
I20260806 18:48:52.319280 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 31349
I20260806 18:48:52.329722 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 30805
2026-08-06T18:48:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1157 ms)
[----------] 6 tests from EncryptionPolicyTest (10820 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:52 dist-test-slave-fgb3 krb5kdc[31612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:52 dist-test-slave-fgb3 krb5kdc[31612](info): set up 2 sockets
Aug 06 18:48:52 dist-test-slave-fgb3 krb5kdc[31612](info): commencing operation
krb5kdc: starting...
W20260806 18:48:53.455869 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:48:53 dist-test-slave-fgb3 krb5kdc[31612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:48:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:48:53.581274 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:53.695458 31633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:53.695732 31633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:53.695794 31633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:53.700166 31633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:53.700233 31633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:53.700261 31633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:53.700284 31633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:53.700320 31633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:53.705612 31633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39147
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:53.706749 31633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:53.707827 31633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:48:53.716602 31633 server_base.cc:1034] running on GCE node
W20260806 18:48:53.716522 31642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:53.716562 31643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:53.716522 31645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:53.717319 31633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:53.718506 31633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:53.719668 31633 hybrid_clock.cc:648] HybridClock initialized: now 1786042133719650 us; error 31 us; skew 500 ppm
Aug 06 18:48:53 dist-test-slave-fgb3 krb5kdc[31612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042133, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:53.722457 31633 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:48:53.723376 31633 webserver.cc:460] Webserver started at http://127.0.0.1:35081/ using document root <none> and password file <none>
I20260806 18:48:53.723925 31633 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:53.723984 31633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:53.724206 31633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:53.726260 31633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "e6e189b7a5d246b9b5670fab31cb5bbd"
format_stamp: "Formatted at 2026-08-06 18:48:53 on dist-test-slave-fgb3"
server_key: "7dc221a154d16839a099539169591d21"
server_key_iv: "872b0f7338b07b57cdf83ec4f386558e"
server_key_version: "encryptionkey@0"
I20260806 18:48:53.726871 31633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "e6e189b7a5d246b9b5670fab31cb5bbd"
format_stamp: "Formatted at 2026-08-06 18:48:53 on dist-test-slave-fgb3"
server_key: "7dc221a154d16839a099539169591d21"
server_key_iv: "872b0f7338b07b57cdf83ec4f386558e"
server_key_version: "encryptionkey@0"
I20260806 18:48:53.730613 31633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:48:53.732960 31654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:53.734119 31633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:48:53.734229 31633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "e6e189b7a5d246b9b5670fab31cb5bbd"
format_stamp: "Formatted at 2026-08-06 18:48:53 on dist-test-slave-fgb3"
server_key: "7dc221a154d16839a099539169591d21"
server_key_iv: "872b0f7338b07b57cdf83ec4f386558e"
server_key_version: "encryptionkey@0"
I20260806 18:48:53.734359 31633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:53.753041 31633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:53.756160 31633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:53.756342 31633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:53.763389 31633 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39147
I20260806 18:48:53.763415 31728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39147 every 8 connection(s)
I20260806 18:48:53.764333 31633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:53.767195 31729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:53.768100 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 31633
I20260806 18:48:53.768224 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:53.768522 2005 external_mini_cluster.cc:1428] Setting key 57e80b8b7efb42138ab379bb4373370b
I20260806 18:48:53.773944 31729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd: Bootstrap starting.
I20260806 18:48:53.776597 31729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:53.777345 31729 log.cc:824] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:48:53 dist-test-slave-fgb3 krb5kdc[31612](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:48:53.779765 31729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd: No bootstrap required, opened a new log
I20260806 18:48:53.783319 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:53.769926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45277 (local address 127.0.0.1:39147)
0806 18:48:53.770387 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:53.770407 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:53.770453 (+ 46us) server_negotiation.cc:407] Connection header received
0806 18:48:53.771267 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:53.771292 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:53.771735 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:53.772113 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:53.772808 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:53.774264 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:53.775148 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:53.775832 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:53.775988 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:53.778690 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:53.778696 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:53.778709 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:53.778736 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:53.780347 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:53.780906 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:53.780910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:53.780914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:53.781014 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:53.781361 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:53.781363 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:53.781365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:53.781740 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:53.781898 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:53.782329 (+ 431us) server_negotiation.cc:299] Negotiation successful
0806 18:48:53.782583 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":115,"threads_started":1}
I20260806 18:48:53.783815 31729 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6e189b7a5d246b9b5670fab31cb5bbd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39147 } }
I20260806 18:48:53.784035 31729 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:53.784094 31729 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6e189b7a5d246b9b5670fab31cb5bbd, State: Initialized, Role: FOLLOWER
I20260806 18:48:53.784595 31729 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6e189b7a5d246b9b5670fab31cb5bbd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39147 } }
I20260806 18:48:53.784754 31729 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:53.784819 31729 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:53.784914 31729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:53.785974 31729 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6e189b7a5d246b9b5670fab31cb5bbd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39147 } }
I20260806 18:48:53.786399 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:48:53.786366 31729 leader_election.cc:302] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6e189b7a5d246b9b5670fab31cb5bbd; no voters:
I20260806 18:48:53.786651 31729 leader_election.cc:288] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:53.786790 31738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:53.787025 31738 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [term 1 LEADER]: Becoming Leader. State: Replica: e6e189b7a5d246b9b5670fab31cb5bbd, State: Running, Role: LEADER
I20260806 18:48:53.787338 31738 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6e189b7a5d246b9b5670fab31cb5bbd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39147 } }
I20260806 18:48:53.787921 31729 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:53.788532 31744 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6e189b7a5d246b9b5670fab31cb5bbd. Latest consensus state: current_term: 1 leader_uuid: "e6e189b7a5d246b9b5670fab31cb5bbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6e189b7a5d246b9b5670fab31cb5bbd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39147 } } }
I20260806 18:48:53.788642 31744 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:53.789294 31743 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6e189b7a5d246b9b5670fab31cb5bbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6e189b7a5d246b9b5670fab31cb5bbd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39147 } } }
I20260806 18:48:53.789368 31757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:53.789391 31743 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:53.792567 31757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:53.798760 31757 catalog_manager.cc:1347] Generated new cluster ID: f4184f8c2f1f4b2bbf296caee6e5aa94
I20260806 18:48:53.798884 31757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:53.799113 31732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:53.788574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49213 (local address 127.0.0.1:39147)
0806 18:48:53.788824 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:53.788828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:53.790096 (+ 1268us) server_negotiation.cc:407] Connection header received
0806 18:48:53.790262 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:53.790265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:53.790330 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:53.790404 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:53.791180 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:53.791954 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:53.793035 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:53.793374 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:53.793474 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:53.795164 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:53.795168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:53.795170 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:53.795196 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:53.796768 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:53.797298 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:53.797301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:53.797303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:53.797398 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:53.797767 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:53.797770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:53.797772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:53.798010 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:53.798172 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:53.798916 (+ 744us) server_negotiation.cc:299] Negotiation successful
0806 18:48:53.799001 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260806 18:48:53.810120 31757 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:53.811071 31757 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:53.816885 31757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6e189b7a5d246b9b5670fab31cb5bbd: Generated new TSK 0
I20260806 18:48:53.817572 31757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:48:53.828007 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 31633
W20260806 18:48:53.838927 31610 connection.cc:535] client connection to 127.0.0.1:39147 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39147): Connection reset by peer (error 104)
2026-08-06T18:48:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:53 dist-test-slave-fgb3 krb5kdc[31776](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:53 dist-test-slave-fgb3 krb5kdc[31776](info): set up 2 sockets
Aug 06 18:48:53 dist-test-slave-fgb3 krb5kdc[31776](info): commencing operation
krb5kdc: starting...
W20260806 18:48:54.951521 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:48:55 dist-test-slave-fgb3 krb5kdc[31776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:48:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:48:55.078845 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:55.193755 31800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:55.194039 31800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:55.194083 31800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:55.198133 31800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:55.198207 31800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:55.198225 31800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:55.198241 31800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:55.198256 31800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:55.203790 31800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35225
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:55.205413 31800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:55.206815 31800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:55.215662 31809 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:55.215672 31808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:55.215806 31800 server_base.cc:1034] running on GCE node
W20260806 18:48:55.215938 31811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:55.216517 31800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:55.217644 31800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:55.218813 31800 hybrid_clock.cc:648] HybridClock initialized: now 1786042135218796 us; error 32 us; skew 500 ppm
Aug 06 18:48:55 dist-test-slave-fgb3 krb5kdc[31776](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042135, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:55.221513 31800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:48:55.222473 31800 webserver.cc:460] Webserver started at http://127.0.0.1:44541/ using document root <none> and password file <none>
I20260806 18:48:55.223021 31800 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:55.223090 31800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:55.223294 31800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:55.225380 31800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "bccf1e83850d4c6d8630dd03e62ec4c6"
format_stamp: "Formatted at 2026-08-06 18:48:55 on dist-test-slave-fgb3"
server_key: "9055d92e9227b1ee4fe5ad8b69b471d0"
server_key_iv: "8a7e284902a9e0a8156e16f251c7b8f7"
server_key_version: "encryptionkey@0"
I20260806 18:48:55.225924 31800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "bccf1e83850d4c6d8630dd03e62ec4c6"
format_stamp: "Formatted at 2026-08-06 18:48:55 on dist-test-slave-fgb3"
server_key: "9055d92e9227b1ee4fe5ad8b69b471d0"
server_key_iv: "8a7e284902a9e0a8156e16f251c7b8f7"
server_key_version: "encryptionkey@0"
I20260806 18:48:55.229594 31800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260806 18:48:55.232089 31818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:55.233296 31800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:48:55.233451 31800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "bccf1e83850d4c6d8630dd03e62ec4c6"
format_stamp: "Formatted at 2026-08-06 18:48:55 on dist-test-slave-fgb3"
server_key: "9055d92e9227b1ee4fe5ad8b69b471d0"
server_key_iv: "8a7e284902a9e0a8156e16f251c7b8f7"
server_key_version: "encryptionkey@0"
I20260806 18:48:55.233548 31800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:55.248800 31800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:55.251857 31800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:55.252038 31800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:55.258993 31800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35225
I20260806 18:48:55.259196 31892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35225 every 8 connection(s)
I20260806 18:48:55.259948 31800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:55.262760 31893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:55.266052 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 31800
I20260806 18:48:55.266196 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:55.266516 2005 external_mini_cluster.cc:1428] Setting key ba7ff304b80d9bc465cf87a1439e5bfa
I20260806 18:48:55.269668 31893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6: Bootstrap starting.
I20260806 18:48:55.272306 31893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:55.273134 31893 log.cc:824] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:55.275310 31893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6: No bootstrap required, opened a new log
I20260806 18:48:55.275624 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:55.268051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53431 (local address 127.0.0.1:35225)
0806 18:48:55.268458 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:55.268473 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:55.268509 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:48:55.269157 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:55.269176 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:55.269515 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:55.269820 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:55.270161 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:55.271219 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:55.272367 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:55.272866 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:55.272988 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:55.273841 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:55.273848 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:55.273865 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:55.274286 (+ 421us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:55.274385 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:55.274413 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:55.274512 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:55.274700 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 18:48:55.274851 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":112,"threads_started":1}
I20260806 18:48:55.278707 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:48:55.279402 31893 raft_consensus.cc:348] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccf1e83850d4c6d8630dd03e62ec4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35225 } }
I20260806 18:48:55.279682 31893 raft_consensus.cc:374] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:55.279752 31893 raft_consensus.cc:729] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bccf1e83850d4c6d8630dd03e62ec4c6, State: Initialized, Role: FOLLOWER
I20260806 18:48:55.280242 31893 consensus_queue.cc:260] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccf1e83850d4c6d8630dd03e62ec4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35225 } }
I20260806 18:48:55.280422 31893 raft_consensus.cc:388] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:55.280493 31893 raft_consensus.cc:482] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:55.280596 31893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:55.281764 31893 raft_consensus.cc:504] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccf1e83850d4c6d8630dd03e62ec4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35225 } }
I20260806 18:48:55.282100 31893 leader_election.cc:302] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bccf1e83850d4c6d8630dd03e62ec4c6; no voters:
I20260806 18:48:55.282442 31893 leader_election.cc:288] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:55.282531 31914 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:55.282747 31914 raft_consensus.cc:686] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [term 1 LEADER]: Becoming Leader. State: Replica: bccf1e83850d4c6d8630dd03e62ec4c6, State: Running, Role: LEADER
I20260806 18:48:55.283051 31914 consensus_queue.cc:237] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccf1e83850d4c6d8630dd03e62ec4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35225 } }
I20260806 18:48:55.283742 31893 sys_catalog.cc:564] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:55.284300 31914 sys_catalog.cc:455] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bccf1e83850d4c6d8630dd03e62ec4c6. Latest consensus state: current_term: 1 leader_uuid: "bccf1e83850d4c6d8630dd03e62ec4c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccf1e83850d4c6d8630dd03e62ec4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35225 } } }
I20260806 18:48:55.284415 31914 sys_catalog.cc:458] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:55.285054 31917 sys_catalog.cc:455] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bccf1e83850d4c6d8630dd03e62ec4c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bccf1e83850d4c6d8630dd03e62ec4c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35225 } } }
I20260806 18:48:55.285154 31917 sys_catalog.cc:458] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:55.285367 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:55.280895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60583 (local address 127.0.0.1:35225)
0806 18:48:55.281018 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:55.281022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:55.281179 (+ 157us) server_negotiation.cc:407] Connection header received
0806 18:48:55.281320 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:55.281322 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:55.281360 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:55.281418 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:55.281696 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:55.282269 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:55.283402 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:55.283754 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:55.283847 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:55.284808 (+ 961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:55.284812 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:55.284815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:55.284829 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:55.284856 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:55.284869 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:55.284932 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:55.285195 (+ 263us) server_negotiation.cc:299] Negotiation successful
0806 18:48:55.285235 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:48:55.285984 31926 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:55.288908 31926 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:55.294301 31926 catalog_manager.cc:1347] Generated new cluster ID: e94ef45ca4334ce5821cad25016ae6b2
I20260806 18:48:55.294411 31926 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:55.311281 31926 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:55.312813 31926 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:55.321605 31926 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bccf1e83850d4c6d8630dd03e62ec4c6: Generated new TSK 0
I20260806 18:48:55.322501 31926 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:48:55.332299 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 31800
W20260806 18:48:55.345652 31774 connection.cc:535] client connection to 127.0.0.1:35225 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35225): Connection reset by peer (error 104)
2026-08-06T18:48:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:55 dist-test-slave-fgb3 krb5kdc[31950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:55 dist-test-slave-fgb3 krb5kdc[31950](info): set up 2 sockets
Aug 06 18:48:55 dist-test-slave-fgb3 krb5kdc[31950](info): commencing operation
krb5kdc: starting...
W20260806 18:48:56.455200 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:48:56 dist-test-slave-fgb3 krb5kdc[31950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:48:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:48:56.579304 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40249
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:56.695323 31966 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:56.695607 31966 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:56.695680 31966 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:56.699744 31966 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:56.699815 31966 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:56.699846 31966 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:56.699882 31966 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:56.699916 31966 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:56.705325 31966 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40249
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31966
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:56.706439 31966 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:56.707494 31966 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:56.716061 31977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:56.716174 31966 server_base.cc:1034] running on GCE node
W20260806 18:48:56.716061 31979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:56.716116 31975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:56.716835 31966 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:56.717905 31966 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:56.719105 31966 hybrid_clock.cc:648] HybridClock initialized: now 1786042136719077 us; error 38 us; skew 500 ppm
Aug 06 18:48:56 dist-test-slave-fgb3 krb5kdc[31950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:56.726692 31966 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:48:56.727752 31966 webserver.cc:460] Webserver started at http://127.0.0.1:44397/ using document root <none> and password file <none>
I20260806 18:48:56.728309 31966 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:56.728381 31966 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:56.728618 31966 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:56.730778 31966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "b3eb01d8bf7f43fab22a590101b5b4eb"
format_stamp: "Formatted at 2026-08-06 18:48:56 on dist-test-slave-fgb3"
server_key: "2a7cc6dcabc7cdc318b241fb04988a54"
server_key_iv: "841fe38a66d7ff43f5689742c6180bd2"
server_key_version: "encryptionkey@0"
I20260806 18:48:56.731328 31966 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "b3eb01d8bf7f43fab22a590101b5b4eb"
format_stamp: "Formatted at 2026-08-06 18:48:56 on dist-test-slave-fgb3"
server_key: "2a7cc6dcabc7cdc318b241fb04988a54"
server_key_iv: "841fe38a66d7ff43f5689742c6180bd2"
server_key_version: "encryptionkey@0"
I20260806 18:48:56.735102 31966 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:48:56.737546 31990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:56.738871 31966 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:48:56.739001 31966 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "b3eb01d8bf7f43fab22a590101b5b4eb"
format_stamp: "Formatted at 2026-08-06 18:48:56 on dist-test-slave-fgb3"
server_key: "2a7cc6dcabc7cdc318b241fb04988a54"
server_key_iv: "841fe38a66d7ff43f5689742c6180bd2"
server_key_version: "encryptionkey@0"
I20260806 18:48:56.739100 31966 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:56.743678 31966 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:56.746691 31966 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:56.746865 31966 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:56.753821 31966 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40249
I20260806 18:48:56.753847 32066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40249 every 8 connection(s)
I20260806 18:48:56.754765 31966 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:56.755695 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 31966
I20260806 18:48:56.755848 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:56.756150 2005 external_mini_cluster.cc:1428] Setting key 0056ecf681ede7e932986bd12eb2a07e
I20260806 18:48:56.758117 32067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:56.761561 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:56.757571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53209 (local address 127.0.0.1:40249)
0806 18:48:56.758024 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:56.758039 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:56.758079 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:48:56.758796 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:56.758817 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:56.759147 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:56.759511 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:56.760444 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:56.760450 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:56.760461 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:56.760472 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:56.760525 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:56.760544 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:56.760669 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:56.760741 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 18:48:56.760888 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":146,"threads_started":1}
I20260806 18:48:56.764407 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:48:56.764632 32067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb: Bootstrap starting.
I20260806 18:48:56.766871 32067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:56.767673 32067 log.cc:824] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:56.767859 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:56.766348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37595 (local address 127.0.0.1:40249)
0806 18:48:56.766465 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:56.766469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:56.766674 (+ 205us) server_negotiation.cc:407] Connection header received
0806 18:48:56.766783 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:56.766786 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:56.766818 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:56.766875 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:56.767505 (+ 630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:56.767509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:56.767511 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:56.767523 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:56.767548 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:56.767558 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:56.767626 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:56.767707 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 18:48:56.767741 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:48:56.769522 32067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb: No bootstrap required, opened a new log
I20260806 18:48:56.772970 32067 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3eb01d8bf7f43fab22a590101b5b4eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } }
I20260806 18:48:56.773182 32067 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:56.773259 32067 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3eb01d8bf7f43fab22a590101b5b4eb, State: Initialized, Role: FOLLOWER
I20260806 18:48:56.773643 32067 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3eb01d8bf7f43fab22a590101b5b4eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } }
I20260806 18:48:56.773768 32067 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:56.773833 32067 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:56.773931 32067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:56.774797 32067 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3eb01d8bf7f43fab22a590101b5b4eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } }
I20260806 18:48:56.775101 32067 leader_election.cc:302] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3eb01d8bf7f43fab22a590101b5b4eb; no voters:
I20260806 18:48:56.775375 32067 leader_election.cc:288] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:56.775504 32085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:56.775748 32085 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [term 1 LEADER]: Becoming Leader. State: Replica: b3eb01d8bf7f43fab22a590101b5b4eb, State: Running, Role: LEADER
I20260806 18:48:56.776062 32085 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3eb01d8bf7f43fab22a590101b5b4eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } }
I20260806 18:48:56.776489 32067 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:56.778059 32086 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3eb01d8bf7f43fab22a590101b5b4eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3eb01d8bf7f43fab22a590101b5b4eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } } }
I20260806 18:48:56.778189 32086 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:56.778535 32087 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3eb01d8bf7f43fab22a590101b5b4eb. Latest consensus state: current_term: 1 leader_uuid: "b3eb01d8bf7f43fab22a590101b5b4eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3eb01d8bf7f43fab22a590101b5b4eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40249 } } }
I20260806 18:48:56.778679 32087 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb [sys.catalog]: This master's current role is: LEADER
W20260806 18:48:56.781415 32106 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:48:56.781484 32106 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:48:56.781540 32097 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:56.782210 32097 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:56.787901 32097 catalog_manager.cc:1347] Generated new cluster ID: 36e0a0a481e34256ac72130dbf1fc8fd
I20260806 18:48:56.787969 32097 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:56.798978 32097 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:56.799300 32097 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:56.803985 32097 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3eb01d8bf7f43fab22a590101b5b4eb: Generated new TSK 0
I20260806 18:48:56.804631 32097 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:48:56.813592 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 31966
2026-08-06T18:48:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1484 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:56 dist-test-slave-fgb3 krb5kdc[32116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:56 dist-test-slave-fgb3 krb5kdc[32116](info): set up 2 sockets
Aug 06 18:48:56 dist-test-slave-fgb3 krb5kdc[32116](info): commencing operation
krb5kdc: starting...
W20260806 18:48:57.931851 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.088s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:48:58 dist-test-slave-fgb3 krb5kdc[32116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:48:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:48:58.055398 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36123
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:58.167138 32134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:58.167407 32134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:58.167448 32134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:58.171360 32134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:58.171409 32134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:58.171427 32134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:58.171442 32134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:58.171455 32134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:58.176590 32134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36123
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44831
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:58.177601 32134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:58.178633 32134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:58.187243 32142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:58.187302 32141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:58.187309 32148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:58.187354 32134 server_base.cc:1034] running on GCE node
I20260806 18:48:58.188198 32134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:58.189414 32134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:58.190574 32134 hybrid_clock.cc:648] HybridClock initialized: now 1786042138190558 us; error 29 us; skew 500 ppm
Aug 06 18:48:58 dist-test-slave-fgb3 krb5kdc[32116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042138, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:58.193400 32134 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:48:58.194363 32134 webserver.cc:460] Webserver started at http://127.0.0.1:39081/ using document root <none> and password file <none>
I20260806 18:48:58.194952 32134 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:58.195024 32134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:58.195225 32134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:58.197309 32134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "72b175a6f68640a4af33e370743c9530"
format_stamp: "Formatted at 2026-08-06 18:48:58 on dist-test-slave-fgb3"
server_key: "734d8f2f51c81d2d67d4ce6afa3ebd40"
server_key_iv: "b02a234d8cb65a5f4a15fd7cda96392c"
server_key_version: "encryptionkey@0"
I20260806 18:48:58.197880 32134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "72b175a6f68640a4af33e370743c9530"
format_stamp: "Formatted at 2026-08-06 18:48:58 on dist-test-slave-fgb3"
server_key: "734d8f2f51c81d2d67d4ce6afa3ebd40"
server_key_iv: "b02a234d8cb65a5f4a15fd7cda96392c"
server_key_version: "encryptionkey@0"
I20260806 18:48:58.201833 32134 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260806 18:48:58.204656 32157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:58.206390 32134 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:48:58.206523 32134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "72b175a6f68640a4af33e370743c9530"
format_stamp: "Formatted at 2026-08-06 18:48:58 on dist-test-slave-fgb3"
server_key: "734d8f2f51c81d2d67d4ce6afa3ebd40"
server_key_iv: "b02a234d8cb65a5f4a15fd7cda96392c"
server_key_version: "encryptionkey@0"
I20260806 18:48:58.206635 32134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:58.219151 32134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:58.222760 32134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:58.222963 32134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:58.230114 32134 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44831
I20260806 18:48:58.230150 32234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44831 every 8 connection(s)
I20260806 18:48:58.231181 32134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:58.232096 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 32134
I20260806 18:48:58.232249 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:58.232620 2005 external_mini_cluster.cc:1428] Setting key 5967a5057be237074dfee440d014976a
I20260806 18:48:58.234915 32235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:58.242657 32235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530: Bootstrap starting.
Aug 06 18:48:58 dist-test-slave-fgb3 krb5kdc[32116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:48:58.245278 32235 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:58.246161 32235 log.cc:824] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:58.248391 32235 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530: No bootstrap required, opened a new log
I20260806 18:48:58.248431 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:58.234268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43751 (local address 127.0.0.1:44831)
0806 18:48:58.234830 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:58.234846 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:58.234885 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:48:58.235634 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:58.235654 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:58.236026 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:58.236365 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:58.237393 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:58.238509 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:58.239687 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:58.240194 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:58.240327 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:58.243486 (+ 3159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:58.243492 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:58.243508 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:58.243541 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:58.245295 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:58.245783 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:58.245789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:58.245794 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:58.245916 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:58.246286 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:58.246290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:58.246291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:58.246795 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:58.246982 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:58.247315 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 18:48:58.247651 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":170,"threads_started":1}
I20260806 18:48:58.251493 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:48:58.252442 32235 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b175a6f68640a4af33e370743c9530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44831 } }
I20260806 18:48:58.252722 32235 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:58.252799 32235 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b175a6f68640a4af33e370743c9530, State: Initialized, Role: FOLLOWER
I20260806 18:48:58.253240 32235 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b175a6f68640a4af33e370743c9530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44831 } }
I20260806 18:48:58.253383 32235 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:58.253448 32235 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:58.253547 32235 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:58.254630 32235 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b175a6f68640a4af33e370743c9530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44831 } }
I20260806 18:48:58.254999 32235 leader_election.cc:302] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72b175a6f68640a4af33e370743c9530; no voters:
I20260806 18:48:58.255347 32235 leader_election.cc:288] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:58.255421 32246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:58.255646 32246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [term 1 LEADER]: Becoming Leader. State: Replica: 72b175a6f68640a4af33e370743c9530, State: Running, Role: LEADER
I20260806 18:48:58.256031 32246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b175a6f68640a4af33e370743c9530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44831 } }
I20260806 18:48:58.256546 32235 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:58.257328 32247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72b175a6f68640a4af33e370743c9530. Latest consensus state: current_term: 1 leader_uuid: "72b175a6f68640a4af33e370743c9530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b175a6f68640a4af33e370743c9530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44831 } } }
I20260806 18:48:58.257517 32247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:58.258005 32262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:58.258046 32248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72b175a6f68640a4af33e370743c9530" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72b175a6f68640a4af33e370743c9530" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44831 } } }
I20260806 18:48:58.258204 32248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:58.261612 32262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:58.262578 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:58.253463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45629 (local address 127.0.0.1:44831)
0806 18:48:58.253585 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:58.253590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:58.253909 (+ 319us) server_negotiation.cc:407] Connection header received
0806 18:48:58.253985 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:58.253988 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:58.254034 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:58.254110 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:58.254741 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:58.255486 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:58.256566 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:58.256893 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:58.257001 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:58.258660 (+ 1659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:58.258664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:58.258666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:48:58.258692 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:58.260239 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:58.260706 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:58.260711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:58.260713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:58.260799 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:48:58.261100 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:48:58.261104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:48:58.261105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:48:58.261316 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:58.261439 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:58.262283 (+ 844us) server_negotiation.cc:299] Negotiation successful
0806 18:48:58.262431 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 18:48:58.267581 32262 catalog_manager.cc:1347] Generated new cluster ID: 718577fed1654aaa80cdc6c216170614
I20260806 18:48:58.267645 32262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:58.292900 32262 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:58.294013 32262 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:58.298779 32262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72b175a6f68640a4af33e370743c9530: Generated new TSK 0
I20260806 18:48:58.299494 32262 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:48:58.308847 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 32134
2026-08-06T18:48:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1496 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:58 dist-test-slave-fgb3 krb5kdc[32275](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:58 dist-test-slave-fgb3 krb5kdc[32275](info): set up 2 sockets
Aug 06 18:48:58 dist-test-slave-fgb3 krb5kdc[32275](info): commencing operation
krb5kdc: starting...
W20260806 18:48:59.445416 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:48:59 dist-test-slave-fgb3 krb5kdc[32275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:48:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:48:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:48:59.570205 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44547
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:48:59.681998 32300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:48:59.682265 32300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:48:59.682356 32300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:48:59.686300 32300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:48:59.686376 32300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:48:59.686400 32300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:48:59.686429 32300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:48:59.686455 32300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:48:59.691653 32300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:48:59.692656 32300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:48:59.693673 32300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:48:59.701895 32306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:59.701900 32305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:48:59.701952 32310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:48:59.702203 32300 server_base.cc:1034] running on GCE node
I20260806 18:48:59.702796 32300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:48:59.703903 32300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:48:59.705080 32300 hybrid_clock.cc:648] HybridClock initialized: now 1786042139705057 us; error 33 us; skew 500 ppm
Aug 06 18:48:59 dist-test-slave-fgb3 krb5kdc[32275](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042139, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:48:59.707913 32300 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:48:59.708845 32300 webserver.cc:460] Webserver started at http://127.0.0.1:34445/ using document root <none> and password file <none>
I20260806 18:48:59.709388 32300 fs_manager.cc:359] Metadata directory not provided
I20260806 18:48:59.709455 32300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:48:59.709657 32300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:48:59.711802 32300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "eac5202afbcd40659b80ab86967320b6"
format_stamp: "Formatted at 2026-08-06 18:48:59 on dist-test-slave-fgb3"
server_key: "f553689ec697e40345d780002b5a1ca0"
server_key_iv: "4baa03d5afb36201d5d12c9cc967fe3f"
server_key_version: "encryptionkey@0"
I20260806 18:48:59.712361 32300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "eac5202afbcd40659b80ab86967320b6"
format_stamp: "Formatted at 2026-08-06 18:48:59 on dist-test-slave-fgb3"
server_key: "f553689ec697e40345d780002b5a1ca0"
server_key_iv: "4baa03d5afb36201d5d12c9cc967fe3f"
server_key_version: "encryptionkey@0"
I20260806 18:48:59.716114 32300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260806 18:48:59.718629 32319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:48:59.719847 32300 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:48:59.719970 32300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "eac5202afbcd40659b80ab86967320b6"
format_stamp: "Formatted at 2026-08-06 18:48:59 on dist-test-slave-fgb3"
server_key: "f553689ec697e40345d780002b5a1ca0"
server_key_iv: "4baa03d5afb36201d5d12c9cc967fe3f"
server_key_version: "encryptionkey@0"
I20260806 18:48:59.720077 32300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:48:59.736109 32300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:48:59.738934 32300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:48:59.739116 32300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:48:59.746064 32300 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44547
I20260806 18:48:59.746086 32402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44547 every 8 connection(s)
I20260806 18:48:59.747071 32300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:48:59.749874 32404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:48:59.755357 32404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6: Bootstrap starting.
I20260806 18:48:59.757390 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 32300
I20260806 18:48:59.757524 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:48:59.757802 2005 external_mini_cluster.cc:1428] Setting key df7942b4ecbdce296ffdaa2a0170368a
I20260806 18:48:59.757843 32404 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6: Neither blocks nor log segments found. Creating new log.
I20260806 18:48:59.758728 32404 log.cc:824] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6: Log is configured to *not* fsync() on all Append() calls
I20260806 18:48:59.760798 32404 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6: No bootstrap required, opened a new log
I20260806 18:48:59.764382 32404 raft_consensus.cc:348] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5202afbcd40659b80ab86967320b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } }
I20260806 18:48:59.764617 32404 raft_consensus.cc:374] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:48:59.764683 32404 raft_consensus.cc:729] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac5202afbcd40659b80ab86967320b6, State: Initialized, Role: FOLLOWER
I20260806 18:48:59.765116 32404 consensus_queue.cc:260] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5202afbcd40659b80ab86967320b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } }
I20260806 18:48:59.765259 32404 raft_consensus.cc:388] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:48:59.765329 32404 raft_consensus.cc:482] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:48:59.765439 32404 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:48:59.766271 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:59.758979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56703 (local address 127.0.0.1:44547)
0806 18:48:59.759481 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:59.759501 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:59.759546 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:48:59.760212 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:59.760230 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:59.760556 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:59.760878 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:59.761279 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:59.762228 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:59.763124 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:59.763597 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:59.763747 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:59.764522 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:59.764527 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:59.764541 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:59.765020 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:59.765078 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:59.765104 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:59.765190 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:59.765315 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 18:48:59.765531 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":187,"threads_started":1}
I20260806 18:48:59.766451 32404 raft_consensus.cc:504] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5202afbcd40659b80ab86967320b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } }
I20260806 18:48:59.766829 32404 leader_election.cc:302] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eac5202afbcd40659b80ab86967320b6; no voters:
I20260806 18:48:59.767196 32404 leader_election.cc:288] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:48:59.767266 32415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:48:59.767486 32415 raft_consensus.cc:686] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [term 1 LEADER]: Becoming Leader. State: Replica: eac5202afbcd40659b80ab86967320b6, State: Running, Role: LEADER
I20260806 18:48:59.767771 32415 consensus_queue.cc:237] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5202afbcd40659b80ab86967320b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } }
I20260806 18:48:59.768496 32404 sys_catalog.cc:564] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:48:59.769381 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:48:59.769419 32417 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eac5202afbcd40659b80ab86967320b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5202afbcd40659b80ab86967320b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } } }
I20260806 18:48:59.769547 32417 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:59.769904 32430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:48:59.769862 32418 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eac5202afbcd40659b80ab86967320b6. Latest consensus state: current_term: 1 leader_uuid: "eac5202afbcd40659b80ab86967320b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac5202afbcd40659b80ab86967320b6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44547 } } }
I20260806 18:48:59.769959 32418 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6 [sys.catalog]: This master's current role is: LEADER
I20260806 18:48:59.773115 32430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:48:59.776443 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:48:59.771647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59313 (local address 127.0.0.1:44547)
0806 18:48:59.771745 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0806 18:48:59.771750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:48:59.772252 (+ 502us) server_negotiation.cc:407] Connection header received
0806 18:48:59.772369 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:48:59.772373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:48:59.772422 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:48:59.772515 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:48:59.772826 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:59.773577 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:59.774690 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:48:59.775014 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:48:59.775136 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:48:59.775977 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:48:59.775982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:48:59.775985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:48:59.776000 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:48:59.776034 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:48:59.776050 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:48:59.776135 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 18:48:59.776241 (+ 106us) server_negotiation.cc:299] Negotiation successful
0806 18:48:59.776297 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 18:48:59.779168 32430 catalog_manager.cc:1347] Generated new cluster ID: d10b0434e5354dabbb9baa5bb242bed8
I20260806 18:48:59.779217 32430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:48:59.804302 32430 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:48:59.805282 32430 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:48:59.810647 32430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eac5202afbcd40659b80ab86967320b6: Generated new TSK 0
I20260806 18:48:59.811311 32430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:48:59.820699 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 32300
2026-08-06T18:48:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1507 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:48:59 dist-test-slave-fgb3 krb5kdc[32456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:48:59 dist-test-slave-fgb3 krb5kdc[32456](info): set up 2 sockets
Aug 06 18:48:59 dist-test-slave-fgb3 krb5kdc[32456](info): commencing operation
krb5kdc: starting...
W20260806 18:49:00.946277 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:01 dist-test-slave-fgb3 krb5kdc[32456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:49:01.069523 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36937
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36937
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:01.182828 32474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:01.183120 32474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:01.183167 32474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:01.187418 32474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:01.187496 32474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:01.187515 32474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:01.187530 32474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:01.187544 32474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:01.193163 32474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36937
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:01.194398 32474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:01.195526 32474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:01.204356 32479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:01.204440 32474 server_base.cc:1034] running on GCE node
W20260806 18:49:01.204371 32481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:01.204370 32483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:01.205120 32474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:01.206454 32474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:01.207641 32474 hybrid_clock.cc:648] HybridClock initialized: now 1786042141207615 us; error 38 us; skew 500 ppm
Aug 06 18:49:01 dist-test-slave-fgb3 krb5kdc[32456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042141, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:01.210820 32474 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:49:01.211912 32474 webserver.cc:460] Webserver started at http://127.0.0.1:35757/ using document root <none> and password file <none>
I20260806 18:49:01.212503 32474 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:01.212567 32474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:01.212785 32474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:01.214991 32474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "a1d828a847c44f918b7ae52bf5a1b1c7"
format_stamp: "Formatted at 2026-08-06 18:49:01 on dist-test-slave-fgb3"
server_key: "77b13d9e5c14c06e84c1b36bcb5116ec"
server_key_iv: "8cb9a1c1873e4ded3db4464bd64f26f6"
server_key_version: "encryptionkey@0"
I20260806 18:49:01.215551 32474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "a1d828a847c44f918b7ae52bf5a1b1c7"
format_stamp: "Formatted at 2026-08-06 18:49:01 on dist-test-slave-fgb3"
server_key: "77b13d9e5c14c06e84c1b36bcb5116ec"
server_key_iv: "8cb9a1c1873e4ded3db4464bd64f26f6"
server_key_version: "encryptionkey@0"
I20260806 18:49:01.219393 32474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:49:01.221879 32492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:01.223141 32474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260806 18:49:01.223254 32474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "a1d828a847c44f918b7ae52bf5a1b1c7"
format_stamp: "Formatted at 2026-08-06 18:49:01 on dist-test-slave-fgb3"
server_key: "77b13d9e5c14c06e84c1b36bcb5116ec"
server_key_iv: "8cb9a1c1873e4ded3db4464bd64f26f6"
server_key_version: "encryptionkey@0"
I20260806 18:49:01.223394 32474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:01.229223 32474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:01.232460 32474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:01.232636 32474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:01.239902 32474 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36937
I20260806 18:49:01.239982 32574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36937 every 8 connection(s)
I20260806 18:49:01.240877 32474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:01.243793 32576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:01.246014 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 32474
I20260806 18:49:01.246141 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:01.246469 2005 external_mini_cluster.cc:1428] Setting key 5d9b17b4763eea44aeeb9941e17b3cc6
I20260806 18:49:01.249886 32576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7: Bootstrap starting.
I20260806 18:49:01.251667 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:01.247790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36071 (local address 127.0.0.1:36937)
0806 18:49:01.248209 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:01.248226 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:01.248262 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:49:01.248940 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:01.248958 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:01.249256 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:01.249533 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:01.250421 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:01.250426 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:01.250438 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:01.250449 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:01.250506 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:01.250526 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:01.250638 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:01.250738 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 18:49:01.250881 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260806 18:49:01.252483 32576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:01.253298 32576 log.cc:824] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:01.254504 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:01.255731 32576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7: No bootstrap required, opened a new log
I20260806 18:49:01.259889 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:01.256820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48449 (local address 127.0.0.1:36937)
0806 18:49:01.256939 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:01.256944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:01.258580 (+ 1636us) server_negotiation.cc:407] Connection header received
0806 18:49:01.258663 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:01.258666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:01.258706 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:01.258806 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:01.259528 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:01.259532 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:01.259535 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:01.259546 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:01.259576 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:01.259588 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:01.259669 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:01.259731 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 18:49:01.259771 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:49:01.259752 32576 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d828a847c44f918b7ae52bf5a1b1c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260806 18:49:01.260058 32576 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:01.260113 32576 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1d828a847c44f918b7ae52bf5a1b1c7, State: Initialized, Role: FOLLOWER
I20260806 18:49:01.260596 32576 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d828a847c44f918b7ae52bf5a1b1c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260806 18:49:01.260761 32576 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:01.260833 32576 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:01.260931 32576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:01.261854 32576 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d828a847c44f918b7ae52bf5a1b1c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260806 18:49:01.262178 32576 leader_election.cc:302] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1d828a847c44f918b7ae52bf5a1b1c7; no voters:
I20260806 18:49:01.262494 32576 leader_election.cc:288] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:01.262595 32587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:01.262799 32587 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [term 1 LEADER]: Becoming Leader. State: Replica: a1d828a847c44f918b7ae52bf5a1b1c7, State: Running, Role: LEADER
I20260806 18:49:01.263177 32587 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d828a847c44f918b7ae52bf5a1b1c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } }
I20260806 18:49:01.263778 32576 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:01.265091 32596 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1d828a847c44f918b7ae52bf5a1b1c7. Latest consensus state: current_term: 1 leader_uuid: "a1d828a847c44f918b7ae52bf5a1b1c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d828a847c44f918b7ae52bf5a1b1c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } } }
I20260806 18:49:01.265105 32588 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1d828a847c44f918b7ae52bf5a1b1c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1d828a847c44f918b7ae52bf5a1b1c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36937 } } }
I20260806 18:49:01.265228 32596 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:01.265236 32588 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:01.265606 32605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:01.268465 32605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:01.273705 32605 catalog_manager.cc:1347] Generated new cluster ID: f1bc2bf008aa4ddda0ad2e7dd9a33908
I20260806 18:49:01.273783 32605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:01.320756 32605 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:01.321128 32605 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:01.325738 32605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1d828a847c44f918b7ae52bf5a1b1c7: Generated new TSK 0
I20260806 18:49:01.326592 32605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:01.335366 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 32474
2026-08-06T18:49:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1512 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:01 dist-test-slave-fgb3 krb5kdc[32625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:01 dist-test-slave-fgb3 krb5kdc[32625](info): set up 2 sockets
Aug 06 18:49:01 dist-test-slave-fgb3 krb5kdc[32625](info): commencing operation
krb5kdc: starting...
W20260806 18:49:02.457397 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:02 dist-test-slave-fgb3 krb5kdc[32625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:49:02.586906 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36381
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:02.697057 32645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:02.697328 32645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:02.697394 32645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:02.701377 32645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:02.701442 32645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:02.701467 32645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:02.701488 32645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:02.701508 32645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:02.706782 32645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:02.707859 32645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:02.708885 32645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:49:02.717303 32645 server_base.cc:1034] running on GCE node
W20260806 18:49:02.717242 32654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:02.717242 32652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:02.717247 32651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:02.718055 32645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:02.719154 32645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:02.720309 32645 hybrid_clock.cc:648] HybridClock initialized: now 1786042142720289 us; error 30 us; skew 500 ppm
Aug 06 18:49:02 dist-test-slave-fgb3 krb5kdc[32625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042142, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:02.726935 32645 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:49:02.727836 32645 webserver.cc:460] Webserver started at http://127.0.0.1:37369/ using document root <none> and password file <none>
I20260806 18:49:02.728370 32645 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:02.728430 32645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:02.728612 32645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:02.730659 32645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "2d0ad1a532044e36a6b72d5297fcbdcf"
format_stamp: "Formatted at 2026-08-06 18:49:02 on dist-test-slave-fgb3"
server_key: "3f9da5facfbc870f61c62e33457f7798"
server_key_iv: "f3db173022a702b85357bf6e75baddb9"
server_key_version: "encryptionkey@0"
I20260806 18:49:02.731211 32645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "2d0ad1a532044e36a6b72d5297fcbdcf"
format_stamp: "Formatted at 2026-08-06 18:49:02 on dist-test-slave-fgb3"
server_key: "3f9da5facfbc870f61c62e33457f7798"
server_key_iv: "f3db173022a702b85357bf6e75baddb9"
server_key_version: "encryptionkey@0"
I20260806 18:49:02.734822 32645 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:49:02.737197 32662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:02.738430 32645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:49:02.738543 32645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "2d0ad1a532044e36a6b72d5297fcbdcf"
format_stamp: "Formatted at 2026-08-06 18:49:02 on dist-test-slave-fgb3"
server_key: "3f9da5facfbc870f61c62e33457f7798"
server_key_iv: "f3db173022a702b85357bf6e75baddb9"
server_key_version: "encryptionkey@0"
I20260806 18:49:02.738634 32645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:02.762856 32645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:02.766062 32645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:02.766244 32645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:02.773087 32645 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36381
I20260806 18:49:02.773103 32739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36381 every 8 connection(s)
I20260806 18:49:02.774000 32645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:02.776708 32741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:02.782109 32741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf: Bootstrap starting.
I20260806 18:49:02.783756 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 32645
I20260806 18:49:02.783903 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:02.784194 2005 external_mini_cluster.cc:1428] Setting key 15b78fd0e596ad254bec04196f555db2
I20260806 18:49:02.784812 32741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:02.785705 32741 log.cc:824] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:02.787810 32741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf: No bootstrap required, opened a new log
I20260806 18:49:02.791973 32741 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0ad1a532044e36a6b72d5297fcbdcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } }
I20260806 18:49:02.792276 32741 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:02.792356 32741 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d0ad1a532044e36a6b72d5297fcbdcf, State: Initialized, Role: FOLLOWER
I20260806 18:49:02.792820 32741 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0ad1a532044e36a6b72d5297fcbdcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } }
I20260806 18:49:02.792948 32741 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:02.793007 32741 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
Aug 06 18:49:02 dist-test-slave-fgb3 krb5kdc[32625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:49:02.793099 32741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:02.794173 32741 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0ad1a532044e36a6b72d5297fcbdcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } }
I20260806 18:49:02.794601 32741 leader_election.cc:302] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d0ad1a532044e36a6b72d5297fcbdcf; no voters:
I20260806 18:49:02.794957 32741 leader_election.cc:288] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:02.795039 32750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:02.795235 32750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [term 1 LEADER]: Becoming Leader. State: Replica: 2d0ad1a532044e36a6b72d5297fcbdcf, State: Running, Role: LEADER
I20260806 18:49:02.795603 32750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0ad1a532044e36a6b72d5297fcbdcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } }
I20260806 18:49:02.796262 32741 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:02.797256 32751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d0ad1a532044e36a6b72d5297fcbdcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0ad1a532044e36a6b72d5297fcbdcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } } }
I20260806 18:49:02.797391 32751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:02.797816 32752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d0ad1a532044e36a6b72d5297fcbdcf. Latest consensus state: current_term: 1 leader_uuid: "2d0ad1a532044e36a6b72d5297fcbdcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d0ad1a532044e36a6b72d5297fcbdcf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36381 } } }
I20260806 18:49:02.797988 32752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:02.799758 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:02.785405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:34425 (local address 127.0.0.1:36381)
0806 18:49:02.785793 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:02.785814 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:02.785856 (+ 42us) server_negotiation.cc:407] Connection header received
0806 18:49:02.786687 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:02.786709 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:02.787043 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:02.787314 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:02.788197 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:02.789175 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:02.790372 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:02.790835 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:02.790966 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:02.793705 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:02.793712 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:02.793729 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:02.793762 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:02.795689 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:02.796257 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:02.796263 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:02.796269 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:02.796412 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:02.796814 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:02.796818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:02.796819 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:02.797352 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:02.798575 (+ 1223us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:02.798760 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 18:49:02.799014 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":99,"threads_started":1}
W20260806 18:49:02.801309 306 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:02.801388 306 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:02.801445 32761 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:02.802261 32761 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:02.803009 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:02.808018 32761 catalog_manager.cc:1347] Generated new cluster ID: 537051a81af5456990f7574b900cbea9
I20260806 18:49:02.808077 32761 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:02.813367 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:02.805060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:55879 (local address 127.0.0.1:36381)
0806 18:49:02.806223 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:02.806227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:02.806242 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:49:02.806275 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:02.806278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:02.806348 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:02.806430 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:02.807326 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:02.808091 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:02.808864 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:02.809100 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:02.809186 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:02.810476 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:02.810479 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:02.810481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:02.810502 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:02.811740 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:02.812183 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:02.812186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:02.812187 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:02.812259 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:02.812556 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:02.812558 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:02.812559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:02.812715 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:02.812852 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:02.813117 (+ 265us) server_negotiation.cc:299] Negotiation successful
0806 18:49:02.813210 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1091}
I20260806 18:49:02.829843 32761 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:02.830761 32761 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:02.840253 32761 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d0ad1a532044e36a6b72d5297fcbdcf: Generated new TSK 0
I20260806 18:49:02.840992 32761 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:02.850656 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 32645
W20260806 18:49:02.862943 32621 connection.cc:535] client connection to 127.0.0.1:36381 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36381): Connection reset by peer (error 104)
2026-08-06T18:49:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1516 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:02 dist-test-slave-fgb3 krb5kdc[321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:02 dist-test-slave-fgb3 krb5kdc[321](info): set up 2 sockets
Aug 06 18:49:02 dist-test-slave-fgb3 krb5kdc[321](info): commencing operation
krb5kdc: starting...
W20260806 18:49:03.967371 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:04 dist-test-slave-fgb3 krb5kdc[321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:49:04.093009 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45373
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45373
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:04.205514 338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:04.205783 338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:04.205823 338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:04.209913 338 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:04.209980 338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:04.209998 338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:04.210012 338 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:04.210026 338 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:04.215430 338 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45373
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45373
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:04.216509 338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:04.217556 338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:04.226212 344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:04.226200 349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:04.226297 338 server_base.cc:1034] running on GCE node
W20260806 18:49:04.226204 353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:04.227052 338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:04.228171 338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:04.229357 338 hybrid_clock.cc:648] HybridClock initialized: now 1786042144229332 us; error 34 us; skew 500 ppm
Aug 06 18:49:04 dist-test-slave-fgb3 krb5kdc[321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042144, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:04.232254 338 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:49:04.233210 338 webserver.cc:460] Webserver started at http://127.0.0.1:46389/ using document root <none> and password file <none>
I20260806 18:49:04.233747 338 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:04.233810 338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:04.233989 338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:04.236469 338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "73ae3a8072934259a4bc16edc28ab085"
format_stamp: "Formatted at 2026-08-06 18:49:04 on dist-test-slave-fgb3"
server_key: "a16ac856c3564705026bb1da83d85cdc"
server_key_iv: "b871847bcacae58af76e6b625229d98a"
server_key_version: "encryptionkey@0"
I20260806 18:49:04.237154 338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "73ae3a8072934259a4bc16edc28ab085"
format_stamp: "Formatted at 2026-08-06 18:49:04 on dist-test-slave-fgb3"
server_key: "a16ac856c3564705026bb1da83d85cdc"
server_key_iv: "b871847bcacae58af76e6b625229d98a"
server_key_version: "encryptionkey@0"
I20260806 18:49:04.241459 338 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260806 18:49:04.244055 362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:04.245434 338 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:49:04.245568 338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "73ae3a8072934259a4bc16edc28ab085"
format_stamp: "Formatted at 2026-08-06 18:49:04 on dist-test-slave-fgb3"
server_key: "a16ac856c3564705026bb1da83d85cdc"
server_key_iv: "b871847bcacae58af76e6b625229d98a"
server_key_version: "encryptionkey@0"
I20260806 18:49:04.245672 338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:04.262620 338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:04.265724 338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:04.265914 338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:04.272920 338 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45373
I20260806 18:49:04.272984 437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45373 every 8 connection(s)
I20260806 18:49:04.273873 338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:04.276813 438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:04.279692 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 338
I20260806 18:49:04.279830 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:04.280082 2005 external_mini_cluster.cc:1428] Setting key 8b40e27ce97c6d2f28419bf0a9f276f6
I20260806 18:49:04.282945 438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085: Bootstrap starting.
I20260806 18:49:04.285744 438 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:04.286598 438 log.cc:824] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:04.288969 438 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085: No bootstrap required, opened a new log
I20260806 18:49:04.290105 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:04.281452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:33299 (local address 127.0.0.1:45373)
0806 18:49:04.281905 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:04.281926 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:04.281970 (+ 44us) server_negotiation.cc:407] Connection header received
0806 18:49:04.282825 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:04.282849 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:04.283299 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:04.283658 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:04.284073 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:04.285571 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:04.286512 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:04.287161 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:04.287294 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:04.288216 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:04.288223 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:04.288239 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:04.288747 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:04.288818 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:04.288849 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:04.288987 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:04.289238 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 18:49:04.289382 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":127,"threads_started":1}
I20260806 18:49:04.293320 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:04.293202 438 raft_consensus.cc:348] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ae3a8072934259a4bc16edc28ab085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } }
I20260806 18:49:04.293481 438 raft_consensus.cc:374] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:04.293537 438 raft_consensus.cc:729] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ae3a8072934259a4bc16edc28ab085, State: Initialized, Role: FOLLOWER
I20260806 18:49:04.293958 438 consensus_queue.cc:260] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ae3a8072934259a4bc16edc28ab085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } }
I20260806 18:49:04.294095 438 raft_consensus.cc:388] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:04.294162 438 raft_consensus.cc:482] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:04.294262 438 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:04.295308 438 raft_consensus.cc:504] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ae3a8072934259a4bc16edc28ab085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } }
I20260806 18:49:04.295687 438 leader_election.cc:302] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 73ae3a8072934259a4bc16edc28ab085; no voters:
I20260806 18:49:04.296041 438 leader_election.cc:288] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:04.296119 457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:04.296389 457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [term 1 LEADER]: Becoming Leader. State: Replica: 73ae3a8072934259a4bc16edc28ab085, State: Running, Role: LEADER
I20260806 18:49:04.296908 457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ae3a8072934259a4bc16edc28ab085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } }
I20260806 18:49:04.297374 438 sys_catalog.cc:564] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:04.298282 457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "73ae3a8072934259a4bc16edc28ab085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ae3a8072934259a4bc16edc28ab085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } } }
I20260806 18:49:04.298445 457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:04.298844 458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 73ae3a8072934259a4bc16edc28ab085. Latest consensus state: current_term: 1 leader_uuid: "73ae3a8072934259a4bc16edc28ab085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "73ae3a8072934259a4bc16edc28ab085" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45373 } } }
I20260806 18:49:04.299002 458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:04.299067 469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:04.300381 443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:04.295473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:37599 (local address 127.0.0.1:45373)
0806 18:49:04.295606 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:04.295611 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:04.295791 (+ 180us) server_negotiation.cc:407] Connection header received
0806 18:49:04.295914 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:04.295917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:04.295974 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:04.296059 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:04.296453 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:04.297190 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:04.298399 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:04.298721 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:04.298795 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:04.299915 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:04.299919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:04.299922 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:04.299937 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:04.299966 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:04.299979 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:04.300057 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:04.300212 (+ 155us) server_negotiation.cc:299] Negotiation successful
0806 18:49:04.300248 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 18:49:04.302456 469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:04.307621 469 catalog_manager.cc:1347] Generated new cluster ID: a1c128de7e964c52a26c231e19f9194f
I20260806 18:49:04.307688 469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:04.314509 469 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:04.315495 469 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:04.320919 469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 73ae3a8072934259a4bc16edc28ab085: Generated new TSK 0
I20260806 18:49:04.321620 469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:04.327575 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 338
W20260806 18:49:04.338934 319 connection.cc:535] client connection to 127.0.0.1:45373 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45373): Connection reset by peer (error 104)
2026-08-06T18:49:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1476 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:04 dist-test-slave-fgb3 krb5kdc[486](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:04 dist-test-slave-fgb3 krb5kdc[486](info): set up 2 sockets
Aug 06 18:49:04 dist-test-slave-fgb3 krb5kdc[486](info): commencing operation
krb5kdc: starting...
W20260806 18:49:05.460095 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:05 dist-test-slave-fgb3 krb5kdc[486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042145, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:49:05.589681 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38511
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:05.701354 502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:05.701643 502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:05.701702 502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:05.705714 502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:05.705785 502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:05.705813 502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:05.705833 502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:05.705853 502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:05.711225 502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:05.712311 502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:05.713335 502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:05.721828 512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:05.721889 502 server_base.cc:1034] running on GCE node
W20260806 18:49:05.721841 513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:05.721841 515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:05.722604 502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:05.723694 502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:05.724864 502 hybrid_clock.cc:648] HybridClock initialized: now 1786042145724847 us; error 25 us; skew 500 ppm
Aug 06 18:49:05 dist-test-slave-fgb3 krb5kdc[486](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042145, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:05.727551 502 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:49:05.728366 502 webserver.cc:460] Webserver started at http://127.0.0.1:41193/ using document root <none> and password file <none>
I20260806 18:49:05.728871 502 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:05.728927 502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:05.729095 502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:05.731069 502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "76134a0e568f48768e0a3dcc98418e03"
format_stamp: "Formatted at 2026-08-06 18:49:05 on dist-test-slave-fgb3"
server_key: "0258bd209f1d00a68a37dc71ec813226"
server_key_iv: "0f6b6354435939efdcc641b32ef1a85c"
server_key_version: "encryptionkey@0"
I20260806 18:49:05.731555 502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "76134a0e568f48768e0a3dcc98418e03"
format_stamp: "Formatted at 2026-08-06 18:49:05 on dist-test-slave-fgb3"
server_key: "0258bd209f1d00a68a37dc71ec813226"
server_key_iv: "0f6b6354435939efdcc641b32ef1a85c"
server_key_version: "encryptionkey@0"
I20260806 18:49:05.734977 502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:49:05.737285 524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:05.738519 502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260806 18:49:05.738624 502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "76134a0e568f48768e0a3dcc98418e03"
format_stamp: "Formatted at 2026-08-06 18:49:05 on dist-test-slave-fgb3"
server_key: "0258bd209f1d00a68a37dc71ec813226"
server_key_iv: "0f6b6354435939efdcc641b32ef1a85c"
server_key_version: "encryptionkey@0"
I20260806 18:49:05.738718 502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:05.743070 502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:05.745981 502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:05.746141 502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:05.752645 502 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38511
I20260806 18:49:05.752709 614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38511 every 8 connection(s)
I20260806 18:49:05.753552 502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:05.756124 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 502
I20260806 18:49:05.756284 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:05.756524 616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:05.756623 2005 external_mini_cluster.cc:1428] Setting key 2872970ab5372a8ca01df65bc6ab180c
I20260806 18:49:05.761914 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:05.757914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:34199 (local address 127.0.0.1:38511)
0806 18:49:05.758378 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:05.758394 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:05.758429 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:49:05.759255 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:05.759273 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:05.759546 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:05.759813 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:05.760640 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:05.760645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:05.760656 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:05.760666 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:05.760718 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:05.760736 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:05.760844 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:05.760959 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 18:49:05.761164 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":110,"threads_started":1}
I20260806 18:49:05.763748 616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03: Bootstrap starting.
I20260806 18:49:05.764865 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:05.766462 616 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:05.767356 616 log.cc:824] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:05.768929 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:05.767174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:56453 (local address 127.0.0.1:38511)
0806 18:49:05.767306 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:05.767310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:05.767559 (+ 249us) server_negotiation.cc:407] Connection header received
0806 18:49:05.767690 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:05.767692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:05.767726 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:05.767808 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:05.768568 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:05.768571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:05.768574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:05.768586 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:05.768614 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:05.768625 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:05.768694 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:05.768769 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 18:49:05.768813 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:49:05.769340 616 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03: No bootstrap required, opened a new log
I20260806 18:49:05.772979 616 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76134a0e568f48768e0a3dcc98418e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38511 } }
I20260806 18:49:05.773216 616 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:05.773275 616 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76134a0e568f48768e0a3dcc98418e03, State: Initialized, Role: FOLLOWER
I20260806 18:49:05.773696 616 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76134a0e568f48768e0a3dcc98418e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38511 } }
I20260806 18:49:05.773862 616 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:05.773932 616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:05.774008 616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:05.774881 616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76134a0e568f48768e0a3dcc98418e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38511 } }
I20260806 18:49:05.775166 616 leader_election.cc:302] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76134a0e568f48768e0a3dcc98418e03; no voters:
I20260806 18:49:05.775406 616 leader_election.cc:288] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:05.775516 634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:05.775734 634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [term 1 LEADER]: Becoming Leader. State: Replica: 76134a0e568f48768e0a3dcc98418e03, State: Running, Role: LEADER
I20260806 18:49:05.776044 634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76134a0e568f48768e0a3dcc98418e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38511 } }
I20260806 18:49:05.776505 616 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:05.777729 637 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76134a0e568f48768e0a3dcc98418e03. Latest consensus state: current_term: 1 leader_uuid: "76134a0e568f48768e0a3dcc98418e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76134a0e568f48768e0a3dcc98418e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38511 } } }
I20260806 18:49:05.777755 635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76134a0e568f48768e0a3dcc98418e03" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76134a0e568f48768e0a3dcc98418e03" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38511 } } }
I20260806 18:49:05.777961 635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:05.777935 637 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03 [sys.catalog]: This master's current role is: LEADER
W20260806 18:49:05.781270 654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:05.781332 654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:05.781407 650 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:05.782063 650 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:05.787556 650 catalog_manager.cc:1347] Generated new cluster ID: d20f6da072fb4c4eabc088273d66724e
I20260806 18:49:05.787616 650 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:05.805322 650 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:05.805671 650 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:05.820273 650 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76134a0e568f48768e0a3dcc98418e03: Generated new TSK 0
I20260806 18:49:05.821031 650 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:05.826393 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 502
2026-08-06T18:49:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1498 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:05 dist-test-slave-fgb3 krb5kdc[664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:05 dist-test-slave-fgb3 krb5kdc[664](info): set up 2 sockets
Aug 06 18:49:05 dist-test-slave-fgb3 krb5kdc[664](info): commencing operation
krb5kdc: starting...
W20260806 18:49:06.960600 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:07 dist-test-slave-fgb3 krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:49:07.090085 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38299
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38299
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:07.203728 681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:07.204007 681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:07.204068 681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:07.208204 681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:07.208282 681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:07.208307 681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:07.208328 681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:07.208348 681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:07.213694 681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38299
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:07.214891 681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:07.215940 681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:07.224471 690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:07.224495 692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:07.224530 681 server_base.cc:1034] running on GCE node
W20260806 18:49:07.224707 694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:07.225317 681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:07.226528 681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:07.227721 681 hybrid_clock.cc:648] HybridClock initialized: now 1786042147227695 us; error 38 us; skew 500 ppm
Aug 06 18:49:07 dist-test-slave-fgb3 krb5kdc[664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042147, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:07.230558 681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:49:07.231498 681 webserver.cc:460] Webserver started at http://127.0.0.1:33337/ using document root <none> and password file <none>
I20260806 18:49:07.232059 681 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:07.232128 681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:07.232347 681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:07.234597 681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "f785f22cebd24c06bc26e0e35e24f9fc"
format_stamp: "Formatted at 2026-08-06 18:49:07 on dist-test-slave-fgb3"
server_key: "5d34bb04ba530ee7d9f2a805541da1bd"
server_key_iv: "5c6884d58385afa5f5058febe693f713"
server_key_version: "encryptionkey@0"
I20260806 18:49:07.235193 681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "f785f22cebd24c06bc26e0e35e24f9fc"
format_stamp: "Formatted at 2026-08-06 18:49:07 on dist-test-slave-fgb3"
server_key: "5d34bb04ba530ee7d9f2a805541da1bd"
server_key_iv: "5c6884d58385afa5f5058febe693f713"
server_key_version: "encryptionkey@0"
I20260806 18:49:07.239231 681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:49:07.241753 702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:07.243079 681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:49:07.243203 681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "f785f22cebd24c06bc26e0e35e24f9fc"
format_stamp: "Formatted at 2026-08-06 18:49:07 on dist-test-slave-fgb3"
server_key: "5d34bb04ba530ee7d9f2a805541da1bd"
server_key_iv: "5c6884d58385afa5f5058febe693f713"
server_key_version: "encryptionkey@0"
I20260806 18:49:07.243305 681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:07.255398 681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:07.258148 681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:07.258335 681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:07.265033 681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38299
I20260806 18:49:07.265123 784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38299 every 8 connection(s)
I20260806 18:49:07.266048 681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:07.266952 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 681
I20260806 18:49:07.267095 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:07.267401 2005 external_mini_cluster.cc:1428] Setting key 771e912e907924cdf3d8822f7e378b97
I20260806 18:49:07.269212 786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:07.274844 786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc: Bootstrap starting.
Aug 06 18:49:07 dist-test-slave-fgb3 krb5kdc[664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 18:49:07.277165 786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:07.277992 786 log.cc:824] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:07.280221 786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc: No bootstrap required, opened a new log
I20260806 18:49:07.281111 790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:07.268720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:60813 (local address 127.0.0.1:38299)
0806 18:49:07.269141 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:07.269160 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:07.269355 (+ 195us) server_negotiation.cc:407] Connection header received
0806 18:49:07.270146 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:07.270164 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:07.270547 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:07.270847 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:07.271624 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:07.272611 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:07.273430 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:07.273901 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:07.274053 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:07.276560 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:07.276565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:07.276577 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:07.276605 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:07.278280 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:07.278815 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:07.278820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:07.278825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:07.278958 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:07.279268 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:07.279270 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:07.279271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:07.279647 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:07.279808 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:07.280119 (+ 311us) server_negotiation.cc:299] Negotiation successful
0806 18:49:07.280329 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":127,"threads_started":1}
I20260806 18:49:07.284204 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:07.284310 786 raft_consensus.cc:348] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f785f22cebd24c06bc26e0e35e24f9fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260806 18:49:07.284548 786 raft_consensus.cc:374] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:07.284605 786 raft_consensus.cc:729] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f785f22cebd24c06bc26e0e35e24f9fc, State: Initialized, Role: FOLLOWER
I20260806 18:49:07.285055 786 consensus_queue.cc:260] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f785f22cebd24c06bc26e0e35e24f9fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260806 18:49:07.285199 786 raft_consensus.cc:388] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:07.285264 786 raft_consensus.cc:482] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:07.285367 786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:07.286406 786 raft_consensus.cc:504] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f785f22cebd24c06bc26e0e35e24f9fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260806 18:49:07.286788 786 leader_election.cc:302] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f785f22cebd24c06bc26e0e35e24f9fc; no voters:
I20260806 18:49:07.287134 786 leader_election.cc:288] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:07.287249 800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:07.287480 800 raft_consensus.cc:686] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [term 1 LEADER]: Becoming Leader. State: Replica: f785f22cebd24c06bc26e0e35e24f9fc, State: Running, Role: LEADER
I20260806 18:49:07.287842 800 consensus_queue.cc:237] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f785f22cebd24c06bc26e0e35e24f9fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } }
I20260806 18:49:07.288465 786 sys_catalog.cc:564] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:07.289003 800 sys_catalog.cc:455] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f785f22cebd24c06bc26e0e35e24f9fc. Latest consensus state: current_term: 1 leader_uuid: "f785f22cebd24c06bc26e0e35e24f9fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f785f22cebd24c06bc26e0e35e24f9fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } } }
I20260806 18:49:07.289176 800 sys_catalog.cc:458] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:07.289965 802 sys_catalog.cc:455] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f785f22cebd24c06bc26e0e35e24f9fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f785f22cebd24c06bc26e0e35e24f9fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38299 } } }
I20260806 18:49:07.290122 802 sys_catalog.cc:458] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc [sys.catalog]: This master's current role is: LEADER
W20260806 18:49:07.293437 823 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:07.293516 823 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:07.293601 810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:07.294461 810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:07.295331 790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:07.286256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:59929 (local address 127.0.0.1:38299)
0806 18:49:07.286394 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:07.286398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:07.286672 (+ 274us) server_negotiation.cc:407] Connection header received
0806 18:49:07.286712 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:07.286715 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:07.286764 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:07.286837 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:07.287683 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:07.288447 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:07.289507 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:07.289845 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:07.289959 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:07.291722 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:07.291726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:07.291729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:07.291753 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:07.293324 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:07.293869 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:07.293873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:07.293874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:07.293964 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:07.294387 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:07.294399 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:07.294401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:07.294617 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:07.294769 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:07.295075 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 18:49:07.295177 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 18:49:07.299538 810 catalog_manager.cc:1347] Generated new cluster ID: 60d3f58b0fb3415ebe1419d129bde50b
I20260806 18:49:07.299604 810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:07.315886 810 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:07.317337 810 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:07.325960 810 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f785f22cebd24c06bc26e0e35e24f9fc: Generated new TSK 0
I20260806 18:49:07.326701 810 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:07.334277 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 681
W20260806 18:49:07.347947 661 connection.cc:535] client connection to 127.0.0.1:38299 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38299): Connection reset by peer (error 104)
2026-08-06T18:49:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1510 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:07 dist-test-slave-fgb3 krb5kdc[830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:07 dist-test-slave-fgb3 krb5kdc[830](info): set up 2 sockets
Aug 06 18:49:07 dist-test-slave-fgb3 krb5kdc[830](info): commencing operation
krb5kdc: starting...
W20260806 18:49:08.475502 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:08 dist-test-slave-fgb3 krb5kdc[830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:49:08.600626 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:08.713958 848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:08.714273 848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:08.714349 848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:08.718609 848 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:08.718689 848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:08.718708 848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:08.718724 848 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:08.718737 848 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:08.724308 848 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.848
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:08.725409 848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:08.726506 848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:49:08.735086 848 server_base.cc:1034] running on GCE node
W20260806 18:49:08.735011 861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:08.735095 858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:08.735028 855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:08.735915 848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:08.737079 848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:08.738276 848 hybrid_clock.cc:648] HybridClock initialized: now 1786042148738258 us; error 27 us; skew 500 ppm
Aug 06 18:49:08 dist-test-slave-fgb3 krb5kdc[830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042148, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:08.745814 848 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:49:08.746924 848 webserver.cc:460] Webserver started at http://127.0.0.1:39709/ using document root <none> and password file <none>
I20260806 18:49:08.747550 848 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:08.747627 848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:08.747871 848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:08.749995 848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "b607c37427c6499399ceee7c369a2535"
format_stamp: "Formatted at 2026-08-06 18:49:08 on dist-test-slave-fgb3"
server_key: "0006acc72eb69578fecb5afce973c056"
server_key_iv: "5d12fd82e2a62b515a994956cbdd5e25"
server_key_version: "encryptionkey@0"
I20260806 18:49:08.750600 848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "b607c37427c6499399ceee7c369a2535"
format_stamp: "Formatted at 2026-08-06 18:49:08 on dist-test-slave-fgb3"
server_key: "0006acc72eb69578fecb5afce973c056"
server_key_iv: "5d12fd82e2a62b515a994956cbdd5e25"
server_key_version: "encryptionkey@0"
I20260806 18:49:08.754482 848 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260806 18:49:08.756800 873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:08.758061 848 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:49:08.758167 848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "b607c37427c6499399ceee7c369a2535"
format_stamp: "Formatted at 2026-08-06 18:49:08 on dist-test-slave-fgb3"
server_key: "0006acc72eb69578fecb5afce973c056"
server_key_iv: "5d12fd82e2a62b515a994956cbdd5e25"
server_key_version: "encryptionkey@0"
I20260806 18:49:08.758252 848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:08.785993 848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:08.788949 848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:08.789115 848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:08.796422 848 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32801
I20260806 18:49:08.796437 955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32801 every 8 connection(s)
I20260806 18:49:08.797407 848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:08.797668 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 848
I20260806 18:49:08.797816 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:08.798168 2005 external_mini_cluster.cc:1428] Setting key 2a2c86ed049cbf52d4e170d6c359ea7c
I20260806 18:49:08.800640 957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:08.807073 957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535: Bootstrap starting.
I20260806 18:49:08.807359 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:08.799620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:43341 (local address 127.0.0.1:32801)
0806 18:49:08.800052 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:08.800069 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:08.800106 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:49:08.800747 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:08.800771 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:08.801119 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:08.801401 (+ 282us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:08.801716 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:08.802756 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:08.803931 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:08.804401 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:08.804517 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:08.805503 (+ 986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:08.805509 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:08.805521 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:08.805930 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:08.805989 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:08.806013 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:08.806116 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:08.806406 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 18:49:08.806558 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":119,"threads_started":1}
I20260806 18:49:08.809543 957 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:08.810434 957 log.cc:824] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:08.810473 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:08.812673 957 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535: No bootstrap required, opened a new log
I20260806 18:49:08.816217 960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:08.812583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:34135 (local address 127.0.0.1:32801)
0806 18:49:08.812701 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:08.812705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:08.812917 (+ 212us) server_negotiation.cc:407] Connection header received
0806 18:49:08.813031 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:08.813034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:08.813075 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:08.813134 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:08.813387 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:08.813875 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:08.814730 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:08.814934 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:08.815019 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:08.815783 (+ 764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:08.815787 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:08.815789 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:08.815800 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:08.815824 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:08.815834 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:08.815899 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:08.816052 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 18:49:08.816089 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:49:08.816282 957 raft_consensus.cc:348] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b607c37427c6499399ceee7c369a2535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32801 } }
I20260806 18:49:08.816581 957 raft_consensus.cc:374] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:08.816644 957 raft_consensus.cc:729] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b607c37427c6499399ceee7c369a2535, State: Initialized, Role: FOLLOWER
I20260806 18:49:08.817067 957 consensus_queue.cc:260] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b607c37427c6499399ceee7c369a2535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32801 } }
I20260806 18:49:08.817231 957 raft_consensus.cc:388] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:08.817301 957 raft_consensus.cc:482] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:08.817394 957 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:08.818341 957 raft_consensus.cc:504] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b607c37427c6499399ceee7c369a2535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32801 } }
I20260806 18:49:08.818679 957 leader_election.cc:302] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b607c37427c6499399ceee7c369a2535; no voters:
I20260806 18:49:08.818974 957 leader_election.cc:288] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:08.819123 972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:08.819391 972 raft_consensus.cc:686] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [term 1 LEADER]: Becoming Leader. State: Replica: b607c37427c6499399ceee7c369a2535, State: Running, Role: LEADER
I20260806 18:49:08.819766 972 consensus_queue.cc:237] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b607c37427c6499399ceee7c369a2535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32801 } }
I20260806 18:49:08.820300 957 sys_catalog.cc:564] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:08.821432 975 sys_catalog.cc:455] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b607c37427c6499399ceee7c369a2535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b607c37427c6499399ceee7c369a2535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32801 } } }
I20260806 18:49:08.821820 975 sys_catalog.cc:458] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:08.821709 976 sys_catalog.cc:455] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b607c37427c6499399ceee7c369a2535. Latest consensus state: current_term: 1 leader_uuid: "b607c37427c6499399ceee7c369a2535" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b607c37427c6499399ceee7c369a2535" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32801 } } }
I20260806 18:49:08.822170 976 sys_catalog.cc:458] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:08.822701 992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:08.825688 992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:08.831729 992 catalog_manager.cc:1347] Generated new cluster ID: c18519a471804334b9bf0fd7dbc273c8
I20260806 18:49:08.831799 992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:08.846061 992 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:08.847571 992 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:08.861488 992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b607c37427c6499399ceee7c369a2535: Generated new TSK 0
I20260806 18:49:08.862507 992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:08.876368 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 848
W20260806 18:49:08.886023 828 connection.cc:535] client connection to 127.0.0.1:32801 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32801): Connection reset by peer (error 104)
2026-08-06T18:49:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1538 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:08 dist-test-slave-fgb3 krb5kdc[1003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:08 dist-test-slave-fgb3 krb5kdc[1003](info): set up 2 sockets
Aug 06 18:49:08 dist-test-slave-fgb3 krb5kdc[1003](info): commencing operation
krb5kdc: starting...
W20260806 18:49:10.001631 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:10 dist-test-slave-fgb3 krb5kdc[1003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 18:49:10.124850 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43053
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:10.240206 1023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:10.240518 1023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:10.240563 1023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:10.244752 1023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:10.244840 1023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:10.244859 1023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:10.244875 1023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:10.244894 1023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:10.250672 1023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43053
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:10.251919 1023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:10.253016 1023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:10.262475 1029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:10.262554 1023 server_base.cc:1034] running on GCE node
W20260806 18:49:10.262435 1032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:10.262493 1035 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:10.263293 1023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:10.264524 1023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:10.265713 1023 hybrid_clock.cc:648] HybridClock initialized: now 1786042150265695 us; error 27 us; skew 500 ppm
Aug 06 18:49:10 dist-test-slave-fgb3 krb5kdc[1003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042150, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:10.268661 1023 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 18:49:10.269557 1023 webserver.cc:460] Webserver started at http://127.0.0.1:42303/ using document root <none> and password file <none>
I20260806 18:49:10.270120 1023 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:10.270190 1023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:10.270434 1023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:10.272568 1023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "1fab40f0d44e44978d373d7401957cc1"
format_stamp: "Formatted at 2026-08-06 18:49:10 on dist-test-slave-fgb3"
server_key: "0a7e570a2f76e4d3ef6f65cfedc9c7ac"
server_key_iv: "d0ae3b924e59f12e17e4f0c93264dd42"
server_key_version: "encryptionkey@0"
I20260806 18:49:10.273123 1023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "1fab40f0d44e44978d373d7401957cc1"
format_stamp: "Formatted at 2026-08-06 18:49:10 on dist-test-slave-fgb3"
server_key: "0a7e570a2f76e4d3ef6f65cfedc9c7ac"
server_key_iv: "d0ae3b924e59f12e17e4f0c93264dd42"
server_key_version: "encryptionkey@0"
I20260806 18:49:10.276945 1023 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:49:10.279472 1046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:10.280709 1023 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:49:10.280829 1023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "1fab40f0d44e44978d373d7401957cc1"
format_stamp: "Formatted at 2026-08-06 18:49:10 on dist-test-slave-fgb3"
server_key: "0a7e570a2f76e4d3ef6f65cfedc9c7ac"
server_key_iv: "d0ae3b924e59f12e17e4f0c93264dd42"
server_key_version: "encryptionkey@0"
I20260806 18:49:10.280972 1023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:10.285606 1023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:10.288756 1023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:10.288942 1023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:10.296005 1023 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43053
I20260806 18:49:10.296041 1125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43053 every 8 connection(s)
I20260806 18:49:10.296967 1023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:10.300060 1126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:10.301404 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 1023
I20260806 18:49:10.301541 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:10.301807 2005 external_mini_cluster.cc:1428] Setting key 20547d20055ccef9c5454fe5c7e3ed86
I20260806 18:49:10.307081 1126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1: Bootstrap starting.
I20260806 18:49:10.307582 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:10.303197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:60867 (local address 127.0.0.1:43053)
0806 18:49:10.303646 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:10.303666 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:10.303713 (+ 47us) server_negotiation.cc:407] Connection header received
0806 18:49:10.304536 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:10.304562 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:10.304955 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:10.305335 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:10.306114 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:10.306119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:10.306134 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:10.306148 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:10.306219 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:10.306248 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:10.306424 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:10.306496 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 18:49:10.306653 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":109,"threads_started":1}
I20260806 18:49:10.309713 1126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:10.310531 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:10.310647 1126 log.cc:824] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:10.313093 1126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1: No bootstrap required, opened a new log
I20260806 18:49:10.314391 1132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:10.312820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.138:48475 (local address 127.0.0.1:43053)
0806 18:49:10.312942 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:10.312946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:10.313090 (+ 144us) server_negotiation.cc:407] Connection header received
0806 18:49:10.313199 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:10.313202 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:10.313239 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:10.313314 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:10.313999 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:10.314003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:10.314006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:10.314018 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:10.314048 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:10.314060 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:10.314147 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:10.314201 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 18:49:10.314240 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:49:10.317247 1126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fab40f0d44e44978d373d7401957cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43053 } }
I20260806 18:49:10.317492 1126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:10.317528 1126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fab40f0d44e44978d373d7401957cc1, State: Initialized, Role: FOLLOWER
I20260806 18:49:10.317974 1126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fab40f0d44e44978d373d7401957cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43053 } }
I20260806 18:49:10.318106 1126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:10.318172 1126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:10.318281 1126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:10.319268 1126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fab40f0d44e44978d373d7401957cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43053 } }
I20260806 18:49:10.319607 1126 leader_election.cc:302] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1fab40f0d44e44978d373d7401957cc1; no voters:
I20260806 18:49:10.319918 1126 leader_election.cc:288] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:10.320019 1141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:10.320225 1141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [term 1 LEADER]: Becoming Leader. State: Replica: 1fab40f0d44e44978d373d7401957cc1, State: Running, Role: LEADER
I20260806 18:49:10.320583 1141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fab40f0d44e44978d373d7401957cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43053 } }
I20260806 18:49:10.321343 1126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:10.322489 1145 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1fab40f0d44e44978d373d7401957cc1. Latest consensus state: current_term: 1 leader_uuid: "1fab40f0d44e44978d373d7401957cc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fab40f0d44e44978d373d7401957cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43053 } } }
I20260806 18:49:10.322535 1144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1fab40f0d44e44978d373d7401957cc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1fab40f0d44e44978d373d7401957cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43053 } } }
I20260806 18:49:10.322613 1145 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:10.322643 1144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:10.322973 1154 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:10.325724 1154 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:10.331753 1154 catalog_manager.cc:1347] Generated new cluster ID: ace8c80caa084b789f9344757e30b899
I20260806 18:49:10.331830 1154 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:10.344806 1154 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:10.345171 1154 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:10.354828 1154 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1fab40f0d44e44978d373d7401957cc1: Generated new TSK 0
I20260806 18:49:10.355684 1154 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:10.360795 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 1023
2026-08-06T18:49:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1488 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:10 dist-test-slave-fgb3 krb5kdc[1173](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:10 dist-test-slave-fgb3 krb5kdc[1173](info): set up 2 sockets
Aug 06 18:49:10 dist-test-slave-fgb3 krb5kdc[1173](info): commencing operation
krb5kdc: starting...
W20260806 18:49:11.488574 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:11 dist-test-slave-fgb3 krb5kdc[1173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:49:11.609995 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:42367
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:42367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:11.724601 1189 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:11.724905 1189 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:11.724969 1189 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:11.729027 1189 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:11.729099 1189 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:11.729126 1189 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:11.729147 1189 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:11.729172 1189 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:11.734637 1189 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:42367
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:42367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1189
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:11.735786 1189 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:11.736830 1189 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:11.745662 1199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:11.745774 1204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:11.745662 1201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:11.745857 1189 server_base.cc:1034] running on GCE node
I20260806 18:49:11.746604 1189 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:11.747823 1189 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:11.749024 1189 hybrid_clock.cc:648] HybridClock initialized: now 1786042151749004 us; error 33 us; skew 500 ppm
Aug 06 18:49:11 dist-test-slave-fgb3 krb5kdc[1173](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042151, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:11.752121 1189 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:49:11.753177 1189 webserver.cc:460] Webserver started at http://127.1.245.126:41621/ using document root <none> and password file <none>
I20260806 18:49:11.753765 1189 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:11.753846 1189 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:11.754061 1189 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:11.756513 1189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "a1f376630dc14c368e6e3056c648d900"
format_stamp: "Formatted at 2026-08-06 18:49:11 on dist-test-slave-fgb3"
server_key: "9bc49c3448bbb6923e924691647c500c"
server_key_iv: "ed745581e1f69f6547088210978ffd3b"
server_key_version: "encryptionkey@0"
I20260806 18:49:11.757131 1189 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "a1f376630dc14c368e6e3056c648d900"
format_stamp: "Formatted at 2026-08-06 18:49:11 on dist-test-slave-fgb3"
server_key: "9bc49c3448bbb6923e924691647c500c"
server_key_iv: "ed745581e1f69f6547088210978ffd3b"
server_key_version: "encryptionkey@0"
I20260806 18:49:11.761193 1189 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:49:11.763882 1212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:11.765197 1189 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:49:11.765332 1189 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "a1f376630dc14c368e6e3056c648d900"
format_stamp: "Formatted at 2026-08-06 18:49:11 on dist-test-slave-fgb3"
server_key: "9bc49c3448bbb6923e924691647c500c"
server_key_iv: "ed745581e1f69f6547088210978ffd3b"
server_key_version: "encryptionkey@0"
I20260806 18:49:11.765437 1189 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:11.787590 1189 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:11.790874 1189 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:11.791042 1189 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:11.798177 1189 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:42367
I20260806 18:49:11.798213 1297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:42367 every 8 connection(s)
I20260806 18:49:11.799165 1189 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:11.801993 1298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:11.806423 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 1189
I20260806 18:49:11.806561 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:11.806820 2005 external_mini_cluster.cc:1428] Setting key b1eeb61e62919cb814b86cbb4e567a26
I20260806 18:49:11.807976 1298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900: Bootstrap starting.
I20260806 18:49:11.810451 1298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:11.811331 1298 log.cc:824] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:11.813491 1298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900: No bootstrap required, opened a new log
Aug 06 18:49:11 dist-test-slave-fgb3 krb5kdc[1173](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:11.817734 1298 raft_consensus.cc:348] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f376630dc14c368e6e3056c648d900" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42367 } }
I20260806 18:49:11.817986 1298 raft_consensus.cc:374] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:11.818032 1298 raft_consensus.cc:729] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1f376630dc14c368e6e3056c648d900, State: Initialized, Role: FOLLOWER
I20260806 18:49:11.818519 1298 consensus_queue.cc:260] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f376630dc14c368e6e3056c648d900" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42367 } }
I20260806 18:49:11.818687 1298 raft_consensus.cc:388] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:11.818760 1298 raft_consensus.cc:482] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:11.818862 1298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:11.819880 1298 raft_consensus.cc:504] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f376630dc14c368e6e3056c648d900" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42367 } }
I20260806 18:49:11.820261 1298 leader_election.cc:302] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a1f376630dc14c368e6e3056c648d900; no voters:
I20260806 18:49:11.820634 1298 leader_election.cc:288] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:11.820729 1304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:11.820950 1304 raft_consensus.cc:686] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [term 1 LEADER]: Becoming Leader. State: Replica: a1f376630dc14c368e6e3056c648d900, State: Running, Role: LEADER
I20260806 18:49:11.821616 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:11.808254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58205 (local address 127.1.245.126:42367)
0806 18:49:11.808697 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:11.808714 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:11.808751 (+ 37us) server_negotiation.cc:407] Connection header received
0806 18:49:11.809421 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:11.809438 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:11.809776 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:11.810071 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:11.811029 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:11.812080 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:11.813177 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:11.813666 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:11.813789 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:11.816634 (+ 2845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:11.816639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:11.816653 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:11.816683 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:11.818449 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:11.818979 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:11.818985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:11.818992 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:11.819112 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:11.819431 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:11.819434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:11.819435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:11.819841 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:11.820015 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:11.820319 (+ 304us) server_negotiation.cc:299] Negotiation successful
0806 18:49:11.820541 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":109,"threads_started":1}
I20260806 18:49:11.821846 1298 sys_catalog.cc:564] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:11.822738 1304 consensus_queue.cc:237] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f376630dc14c368e6e3056c648d900" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42367 } }
I20260806 18:49:11.824460 1306 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a1f376630dc14c368e6e3056c648d900. Latest consensus state: current_term: 1 leader_uuid: "a1f376630dc14c368e6e3056c648d900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f376630dc14c368e6e3056c648d900" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42367 } } }
I20260806 18:49:11.824613 1305 sys_catalog.cc:455] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a1f376630dc14c368e6e3056c648d900" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a1f376630dc14c368e6e3056c648d900" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 42367 } } }
I20260806 18:49:11.824643 1306 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:11.824672 1305 sys_catalog.cc:458] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:11.825048 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 18:49:11.827790 1323 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:11.827880 1323 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:11.827941 1324 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:11.828874 1324 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:11.835469 1324 catalog_manager.cc:1347] Generated new cluster ID: 859155e1f1604c7390e7e959764bc203
I20260806 18:49:11.835546 1324 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:11.839959 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:11.830830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52025 (local address 127.1.245.126:42367)
0806 18:49:11.830941 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:11.830946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:11.831147 (+ 201us) server_negotiation.cc:407] Connection header received
0806 18:49:11.831283 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:11.831287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:11.831335 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:11.831423 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:11.832268 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:11.833454 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:11.834286 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:11.834595 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:11.834687 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:11.836442 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:11.836447 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:11.836450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:11.836478 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:11.838128 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:11.838699 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:11.838703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:11.838705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:11.838799 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:11.839087 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:11.839090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:11.839092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:11.839319 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:11.839461 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:11.839692 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:49:11.839797 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 18:49:11.847733 1324 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:11.848628 1324 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:11.854249 1324 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a1f376630dc14c368e6e3056c648d900: Generated new TSK 0
I20260806 18:49:11.854995 1324 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:11.862308 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 1189
W20260806 18:49:11.878567 1171 connection.cc:535] client connection to 127.1.245.126:42367 recv error: Network error: failed to read from TLS socket (remote: 127.1.245.126:42367): Connection reset by peer (error 104)
2026-08-06T18:49:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:11 dist-test-slave-fgb3 krb5kdc[1341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:11 dist-test-slave-fgb3 krb5kdc[1341](info): set up 2 sockets
Aug 06 18:49:11 dist-test-slave-fgb3 krb5kdc[1341](info): commencing operation
krb5kdc: starting...
W20260806 18:49:12.977371 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:13 dist-test-slave-fgb3 krb5kdc[1341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:49:13.102524 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:39991
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:46505
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:39991
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:13.214893 1365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:13.215178 1365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:13.215220 1365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:13.219234 1365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:13.219295 1365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:13.219312 1365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:13.219327 1365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:13.219340 1365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:13.224677 1365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46505
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:39991
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:39991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:13.225765 1365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:13.226792 1365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:13.235349 1371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:13.235371 1370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:13.235344 1373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:13.235822 1365 server_base.cc:1034] running on GCE node
I20260806 18:49:13.236405 1365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:13.237509 1365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:13.238685 1365 hybrid_clock.cc:648] HybridClock initialized: now 1786042153238664 us; error 28 us; skew 500 ppm
Aug 06 18:49:13 dist-test-slave-fgb3 krb5kdc[1341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042153, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:13.241576 1365 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:49:13.242503 1365 webserver.cc:460] Webserver started at http://127.1.245.126:45175/ using document root <none> and password file <none>
I20260806 18:49:13.243019 1365 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:13.243077 1365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:13.243243 1365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:13.245241 1365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "df4ebbc256ce496eb23eb66f6bb1f89c"
format_stamp: "Formatted at 2026-08-06 18:49:13 on dist-test-slave-fgb3"
server_key: "f235f9348f8e717c29a54a1aa86d8fa4"
server_key_iv: "efcbc5d2aa1c308d4004c7fad8dc2c3b"
server_key_version: "encryptionkey@0"
I20260806 18:49:13.245754 1365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "df4ebbc256ce496eb23eb66f6bb1f89c"
format_stamp: "Formatted at 2026-08-06 18:49:13 on dist-test-slave-fgb3"
server_key: "f235f9348f8e717c29a54a1aa86d8fa4"
server_key_iv: "efcbc5d2aa1c308d4004c7fad8dc2c3b"
server_key_version: "encryptionkey@0"
I20260806 18:49:13.249347 1365 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260806 18:49:13.251765 1383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:13.252964 1365 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 18:49:13.253078 1365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "df4ebbc256ce496eb23eb66f6bb1f89c"
format_stamp: "Formatted at 2026-08-06 18:49:13 on dist-test-slave-fgb3"
server_key: "f235f9348f8e717c29a54a1aa86d8fa4"
server_key_iv: "efcbc5d2aa1c308d4004c7fad8dc2c3b"
server_key_version: "encryptionkey@0"
I20260806 18:49:13.253170 1365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:13.264050 1365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:13.266667 1365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:13.266860 1365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:13.274024 1365 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:39991
I20260806 18:49:13.274081 1462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:39991 every 8 connection(s)
I20260806 18:49:13.275015 1365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:13.277781 1465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:13.279067 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 1365
I20260806 18:49:13.279186 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:13.279443 2005 external_mini_cluster.cc:1428] Setting key d81fd31ea5a45b56038f60308247a58e
I20260806 18:49:13.284267 1465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c: Bootstrap starting.
I20260806 18:49:13.286929 1465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:13.287791 1465 log.cc:824] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:13.289168 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:13.280662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40829 (local address 127.1.245.126:39991)
0806 18:49:13.281075 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:13.281095 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:13.281225 (+ 130us) server_negotiation.cc:407] Connection header received
0806 18:49:13.282110 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:13.282134 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:13.282602 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:13.282966 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:13.283398 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:13.284788 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:13.285661 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:13.286372 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:13.286533 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:13.287296 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:13.287302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:13.287318 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:13.287820 (+ 502us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:13.287891 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:13.287925 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:13.288049 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:13.288221 (+ 172us) server_negotiation.cc:299] Negotiation successful
0806 18:49:13.288342 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":101,"threads_started":1}
I20260806 18:49:13.290068 1465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c: No bootstrap required, opened a new log
I20260806 18:49:13.291986 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:13.294245 1465 raft_consensus.cc:348] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4ebbc256ce496eb23eb66f6bb1f89c" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39991 } }
I20260806 18:49:13.294555 1465 raft_consensus.cc:374] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:13.294661 1465 raft_consensus.cc:729] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df4ebbc256ce496eb23eb66f6bb1f89c, State: Initialized, Role: FOLLOWER
I20260806 18:49:13.295186 1465 consensus_queue.cc:260] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4ebbc256ce496eb23eb66f6bb1f89c" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39991 } }
I20260806 18:49:13.295359 1465 raft_consensus.cc:388] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:13.295430 1465 raft_consensus.cc:482] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:13.295528 1465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:13.296562 1465 raft_consensus.cc:504] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4ebbc256ce496eb23eb66f6bb1f89c" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39991 } }
I20260806 18:49:13.296952 1465 leader_election.cc:302] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df4ebbc256ce496eb23eb66f6bb1f89c; no voters:
I20260806 18:49:13.297341 1465 leader_election.cc:288] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:13.297430 1480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:13.297652 1480 raft_consensus.cc:686] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [term 1 LEADER]: Becoming Leader. State: Replica: df4ebbc256ce496eb23eb66f6bb1f89c, State: Running, Role: LEADER
I20260806 18:49:13.298259 1480 consensus_queue.cc:237] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4ebbc256ce496eb23eb66f6bb1f89c" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39991 } }
I20260806 18:49:13.298471 1469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:13.294285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45033 (local address 127.1.245.126:39991)
0806 18:49:13.294421 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:13.294426 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:13.294596 (+ 170us) server_negotiation.cc:407] Connection header received
0806 18:49:13.294718 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:13.294721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:13.294772 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:13.294853 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:13.295119 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:13.295854 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:13.296633 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:13.297008 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:13.297108 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:13.297915 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:13.297919 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:13.297921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:13.297935 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:13.297973 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:13.297986 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:13.298073 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:13.298227 (+ 154us) server_negotiation.cc:299] Negotiation successful
0806 18:49:13.298262 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 18:49:13.298869 1465 sys_catalog.cc:564] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:13.300134 1486 sys_catalog.cc:455] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [sys.catalog]: SysCatalogTable state changed. Reason: New leader df4ebbc256ce496eb23eb66f6bb1f89c. Latest consensus state: current_term: 1 leader_uuid: "df4ebbc256ce496eb23eb66f6bb1f89c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4ebbc256ce496eb23eb66f6bb1f89c" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39991 } } }
I20260806 18:49:13.300551 1486 sys_catalog.cc:458] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:13.300448 1481 sys_catalog.cc:455] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df4ebbc256ce496eb23eb66f6bb1f89c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df4ebbc256ce496eb23eb66f6bb1f89c" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 39991 } } }
I20260806 18:49:13.301127 1481 sys_catalog.cc:458] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c [sys.catalog]: This master's current role is: LEADER
W20260806 18:49:13.303413 1507 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:13.303481 1507 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:13.303612 1506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:13.304576 1506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:13.310155 1506 catalog_manager.cc:1347] Generated new cluster ID: c33f9f17db5c4509846201b930b101a1
I20260806 18:49:13.310230 1506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:13.334141 1506 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:13.335201 1506 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:13.341780 1506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df4ebbc256ce496eb23eb66f6bb1f89c: Generated new TSK 0
I20260806 18:49:13.342581 1506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:13.357168 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 1365
W20260806 18:49:13.374224 1337 connection.cc:535] client connection to 127.1.245.126:39991 recv error: Network error: failed to read from TLS socket (remote: 127.1.245.126:39991): Connection reset by peer (error 104)
2026-08-06T18:49:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1495 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:13 dist-test-slave-fgb3 krb5kdc[1517](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:13 dist-test-slave-fgb3 krb5kdc[1517](info): set up 2 sockets
Aug 06 18:49:13 dist-test-slave-fgb3 krb5kdc[1517](info): commencing operation
krb5kdc: starting...
W20260806 18:49:14.498767 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:14 dist-test-slave-fgb3 krb5kdc[1517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042154, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:49:14.620955 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:45883
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:45883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:14.732292 1537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:14.732550 1537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:14.732590 1537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:14.736543 1537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:14.736609 1537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:14.736629 1537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:14.736644 1537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:14.736657 1537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:14.742059 1537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:45883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:45883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:14.743256 1537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:14.744303 1537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:14.752962 1548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:14.752977 1543 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:14.752977 1544 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:14.753347 1537 server_base.cc:1034] running on GCE node
I20260806 18:49:14.753882 1537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:14.755043 1537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:14.756250 1537 hybrid_clock.cc:648] HybridClock initialized: now 1786042154756198 us; error 32 us; skew 500 ppm
Aug 06 18:49:14 dist-test-slave-fgb3 krb5kdc[1517](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042154, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:14.763496 1537 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:49:14.764432 1537 webserver.cc:460] Webserver started at http://127.1.245.126:43665/ using document root <none> and password file <none>
I20260806 18:49:14.764978 1537 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:14.765046 1537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:14.765256 1537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:14.767417 1537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "c03c3e3e5ba74231babac4b5cc643d47"
format_stamp: "Formatted at 2026-08-06 18:49:14 on dist-test-slave-fgb3"
server_key: "a999e96b0a484e9403f2b3c62c9946d0"
server_key_iv: "09b848e8003fd419365f2e5815abe86a"
server_key_version: "encryptionkey@0"
I20260806 18:49:14.767953 1537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "c03c3e3e5ba74231babac4b5cc643d47"
format_stamp: "Formatted at 2026-08-06 18:49:14 on dist-test-slave-fgb3"
server_key: "a999e96b0a484e9403f2b3c62c9946d0"
server_key_iv: "09b848e8003fd419365f2e5815abe86a"
server_key_version: "encryptionkey@0"
I20260806 18:49:14.771596 1537 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260806 18:49:14.775465 1559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:14.776786 1537 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260806 18:49:14.776898 1537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "c03c3e3e5ba74231babac4b5cc643d47"
format_stamp: "Formatted at 2026-08-06 18:49:14 on dist-test-slave-fgb3"
server_key: "a999e96b0a484e9403f2b3c62c9946d0"
server_key_iv: "09b848e8003fd419365f2e5815abe86a"
server_key_version: "encryptionkey@0"
I20260806 18:49:14.777010 1537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:14.781672 1537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:14.784507 1537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:14.784648 1537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:14.791690 1537 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:45883
I20260806 18:49:14.791718 1637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:45883 every 8 connection(s)
I20260806 18:49:14.792634 1537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:14.795557 1638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:14.797508 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 1537
I20260806 18:49:14.797623 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:14.797896 2005 external_mini_cluster.cc:1428] Setting key 83b3c341206264be29d899ec06b36cfa
I20260806 18:49:14.802290 1638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47: Bootstrap starting.
I20260806 18:49:14.803073 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:14.799322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40635 (local address 127.1.245.126:45883)
0806 18:49:14.799719 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:14.799735 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:14.799774 (+ 39us) server_negotiation.cc:407] Connection header received
0806 18:49:14.800443 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:14.800462 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:14.800737 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:14.801005 (+ 268us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:14.801824 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:14.801829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:14.801841 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:14.801852 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:14.801906 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:14.801925 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:14.802031 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:14.802127 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 18:49:14.802299 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":107,"threads_started":1}
I20260806 18:49:14.804591 1638 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:14.805379 1638 log.cc:824] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:14.805649 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:14.807623 1638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47: No bootstrap required, opened a new log
I20260806 18:49:14.809399 1642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:14.807757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47697 (local address 127.1.245.126:45883)
0806 18:49:14.807856 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:14.807860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:14.808113 (+ 253us) server_negotiation.cc:407] Connection header received
0806 18:49:14.808254 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:14.808258 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:14.808290 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:14.808361 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:14.809023 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:14.809026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:14.809028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:14.809038 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:14.809061 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:14.809076 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:14.809141 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:14.809250 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 18:49:14.809281 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:49:14.811880 1638 raft_consensus.cc:348] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03c3e3e5ba74231babac4b5cc643d47" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45883 } }
I20260806 18:49:14.812170 1638 raft_consensus.cc:374] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:14.812234 1638 raft_consensus.cc:729] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c03c3e3e5ba74231babac4b5cc643d47, State: Initialized, Role: FOLLOWER
I20260806 18:49:14.812646 1638 consensus_queue.cc:260] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03c3e3e5ba74231babac4b5cc643d47" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45883 } }
I20260806 18:49:14.812769 1638 raft_consensus.cc:388] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:14.812814 1638 raft_consensus.cc:482] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:14.812883 1638 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:14.813726 1638 raft_consensus.cc:504] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03c3e3e5ba74231babac4b5cc643d47" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45883 } }
I20260806 18:49:14.814019 1638 leader_election.cc:302] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c03c3e3e5ba74231babac4b5cc643d47; no voters:
I20260806 18:49:14.814278 1638 leader_election.cc:288] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:14.814388 1650 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:14.814611 1650 raft_consensus.cc:686] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [term 1 LEADER]: Becoming Leader. State: Replica: c03c3e3e5ba74231babac4b5cc643d47, State: Running, Role: LEADER
I20260806 18:49:14.814951 1650 consensus_queue.cc:237] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03c3e3e5ba74231babac4b5cc643d47" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45883 } }
I20260806 18:49:14.815512 1638 sys_catalog.cc:564] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:14.816532 1653 sys_catalog.cc:455] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c03c3e3e5ba74231babac4b5cc643d47. Latest consensus state: current_term: 1 leader_uuid: "c03c3e3e5ba74231babac4b5cc643d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03c3e3e5ba74231babac4b5cc643d47" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45883 } } }
I20260806 18:49:14.816653 1653 sys_catalog.cc:458] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:14.817025 1662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:14.817016 1652 sys_catalog.cc:455] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c03c3e3e5ba74231babac4b5cc643d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c03c3e3e5ba74231babac4b5cc643d47" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 45883 } } }
I20260806 18:49:14.817226 1652 sys_catalog.cc:458] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:14.819943 1662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:14.825531 1662 catalog_manager.cc:1347] Generated new cluster ID: b9bc551cb2da47ddbdaf503b81402b26
I20260806 18:49:14.825598 1662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:14.848851 1662 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:14.849205 1662 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:14.859390 1662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c03c3e3e5ba74231babac4b5cc643d47: Generated new TSK 0
I20260806 18:49:14.860061 1662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:14.872429 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 1537
2026-08-06T18:49:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1506 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:14 dist-test-slave-fgb3 krb5kdc[1683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:14 dist-test-slave-fgb3 krb5kdc[1683](info): set up 2 sockets
Aug 06 18:49:14 dist-test-slave-fgb3 krb5kdc[1683](info): commencing operation
krb5kdc: starting...
W20260806 18:49:15.987133 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:16 dist-test-slave-fgb3 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:49:16.113418 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:37873
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:43603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:37873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:16.226506 1701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:16.226814 1701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:16.226876 1701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:16.231022 1701 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:16.231103 1701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:16.231130 1701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:16.231151 1701 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:16.231181 1701 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:16.236719 1701 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:43603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:37873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:37873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:16.237975 1701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:16.239125 1701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:16.248080 1706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:16.248093 1710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:16.248080 1707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:16.248265 1701 server_base.cc:1034] running on GCE node
I20260806 18:49:16.248967 1701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:16.250101 1701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:16.251260 1701 hybrid_clock.cc:648] HybridClock initialized: now 1786042156251246 us; error 54 us; skew 500 ppm
Aug 06 18:49:16 dist-test-slave-fgb3 krb5kdc[1683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042156, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:16.254037 1701 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:49:16.255012 1701 webserver.cc:460] Webserver started at http://127.1.245.126:41099/ using document root <none> and password file <none>
I20260806 18:49:16.255555 1701 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:16.255611 1701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:16.255786 1701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:16.257822 1701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "d80b00f670634ca0a14d71f488a1fa30"
format_stamp: "Formatted at 2026-08-06 18:49:16 on dist-test-slave-fgb3"
server_key: "8f16147433ab8a82790f29117025a277"
server_key_iv: "cff29ecd9af2fb0b2e9e12642cf199fa"
server_key_version: "encryptionkey@0"
I20260806 18:49:16.258514 1701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "d80b00f670634ca0a14d71f488a1fa30"
format_stamp: "Formatted at 2026-08-06 18:49:16 on dist-test-slave-fgb3"
server_key: "8f16147433ab8a82790f29117025a277"
server_key_iv: "cff29ecd9af2fb0b2e9e12642cf199fa"
server_key_version: "encryptionkey@0"
I20260806 18:49:16.262306 1701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260806 18:49:16.264789 1717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:16.266109 1701 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:49:16.266216 1701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "d80b00f670634ca0a14d71f488a1fa30"
format_stamp: "Formatted at 2026-08-06 18:49:16 on dist-test-slave-fgb3"
server_key: "8f16147433ab8a82790f29117025a277"
server_key_iv: "cff29ecd9af2fb0b2e9e12642cf199fa"
server_key_version: "encryptionkey@0"
I20260806 18:49:16.266304 1701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:16.286576 1701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:16.289726 1701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:16.289927 1701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:16.297256 1701 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:37873
I20260806 18:49:16.297317 1795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:37873 every 8 connection(s)
I20260806 18:49:16.298280 1701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:16.300377 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 1701
I20260806 18:49:16.300526 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:16.300855 2005 external_mini_cluster.cc:1428] Setting key a53c3e5e1981a0a85325033b5a0f885d
I20260806 18:49:16.301580 1797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:16.308267 1797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30: Bootstrap starting.
Aug 06 18:49:16 dist-test-slave-fgb3 krb5kdc[1683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:16.311167 1797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:16.312007 1797 log.cc:824] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:16.314539 1797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30: No bootstrap required, opened a new log
I20260806 18:49:16.315359 1802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:16.302356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51655 (local address 127.1.245.126:37873)
0806 18:49:16.302801 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:16.302817 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:16.302853 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:49:16.303543 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:16.303561 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:16.303899 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:16.304204 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:16.305023 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:16.306027 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:16.306895 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:16.307359 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:16.307514 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:16.310375 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:16.310382 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:16.310403 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:16.310434 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:16.312115 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:16.312569 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:16.312575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:16.312581 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:16.312714 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:16.313015 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:16.313019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:16.313021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:16.313571 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:16.313819 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:16.313990 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 18:49:16.314352 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":150,"threads_started":1}
I20260806 18:49:16.318183 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:16.318759 1797 raft_consensus.cc:348] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80b00f670634ca0a14d71f488a1fa30" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37873 } }
I20260806 18:49:16.319092 1797 raft_consensus.cc:374] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:16.319166 1797 raft_consensus.cc:729] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d80b00f670634ca0a14d71f488a1fa30, State: Initialized, Role: FOLLOWER
I20260806 18:49:16.319682 1797 consensus_queue.cc:260] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80b00f670634ca0a14d71f488a1fa30" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37873 } }
I20260806 18:49:16.319877 1797 raft_consensus.cc:388] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:16.319962 1797 raft_consensus.cc:482] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:16.320084 1797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:16.321354 1797 raft_consensus.cc:504] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80b00f670634ca0a14d71f488a1fa30" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37873 } }
I20260806 18:49:16.321713 1797 leader_election.cc:302] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d80b00f670634ca0a14d71f488a1fa30; no voters:
I20260806 18:49:16.322033 1797 leader_election.cc:288] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:16.322135 1811 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:16.322372 1811 raft_consensus.cc:686] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [term 1 LEADER]: Becoming Leader. State: Replica: d80b00f670634ca0a14d71f488a1fa30, State: Running, Role: LEADER
I20260806 18:49:16.322783 1811 consensus_queue.cc:237] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80b00f670634ca0a14d71f488a1fa30" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37873 } }
I20260806 18:49:16.323529 1797 sys_catalog.cc:564] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:16.324072 1811 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d80b00f670634ca0a14d71f488a1fa30. Latest consensus state: current_term: 1 leader_uuid: "d80b00f670634ca0a14d71f488a1fa30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80b00f670634ca0a14d71f488a1fa30" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37873 } } }
I20260806 18:49:16.324728 1812 sys_catalog.cc:455] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d80b00f670634ca0a14d71f488a1fa30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d80b00f670634ca0a14d71f488a1fa30" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 37873 } } }
I20260806 18:49:16.324862 1811 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:16.324924 1812 sys_catalog.cc:458] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:16.325359 1827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:16.329749 1827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:16.330135 1802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:16.320681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42733 (local address 127.1.245.126:37873)
0806 18:49:16.320829 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:16.320834 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:16.320984 (+ 150us) server_negotiation.cc:407] Connection header received
0806 18:49:16.321082 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:16.321086 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:16.321142 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:16.321241 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:16.321920 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:16.322709 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:16.323590 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:16.323872 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:16.323967 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:16.326518 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:16.326523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:16.326525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:16.326554 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:16.328319 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:16.328814 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:16.328819 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:16.328820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:16.328925 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:16.329212 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:16.329215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:16.329217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:16.329454 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:16.329603 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:16.329844 (+ 241us) server_negotiation.cc:299] Negotiation successful
0806 18:49:16.329971 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 18:49:16.335987 1827 catalog_manager.cc:1347] Generated new cluster ID: 402f152c41494de186b4c575db575d50
I20260806 18:49:16.336076 1827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:16.347802 1827 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:16.349258 1827 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:16.358284 1827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d80b00f670634ca0a14d71f488a1fa30: Generated new TSK 0
I20260806 18:49:16.359217 1827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:16.366058 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 1701
2026-08-06T18:49:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1499 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:16 dist-test-slave-fgb3 krb5kdc[1841](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:16 dist-test-slave-fgb3 krb5kdc[1841](info): set up 2 sockets
Aug 06 18:49:16 dist-test-slave-fgb3 krb5kdc[1841](info): commencing operation
krb5kdc: starting...
W20260806 18:49:17.497974 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:17 dist-test-slave-fgb3 krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:49:17.625187 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35511
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:46589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35511
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:17.739379 1855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:17.739672 1855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:17.739715 1855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:17.743781 1855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:17.743850 1855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:17.743870 1855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:17.743884 1855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:17.743898 1855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:17.749300 1855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:46589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35511
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35511
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:17.750484 1855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:17.751569 1855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:17.760196 1862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:17.760207 1860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:17.760250 1855 server_base.cc:1034] running on GCE node
W20260806 18:49:17.760210 1868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:17.761065 1855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:17.762238 1855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:17.763392 1855 hybrid_clock.cc:648] HybridClock initialized: now 1786042157763367 us; error 38 us; skew 500 ppm
Aug 06 18:49:17 dist-test-slave-fgb3 krb5kdc[1841](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042157, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:17.766177 1855 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:49:17.767184 1855 webserver.cc:460] Webserver started at http://127.1.245.126:34987/ using document root <none> and password file <none>
I20260806 18:49:17.767751 1855 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:17.767820 1855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:17.768033 1855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:17.770126 1855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "ac3f41d0c0a249dbbd308d29e79e51f1"
format_stamp: "Formatted at 2026-08-06 18:49:17 on dist-test-slave-fgb3"
server_key: "82b79bf141f70dd252ce8cc584f60f93"
server_key_iv: "3c2f5023528575815edd083c4c6b9646"
server_key_version: "encryptionkey@0"
I20260806 18:49:17.770727 1855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "ac3f41d0c0a249dbbd308d29e79e51f1"
format_stamp: "Formatted at 2026-08-06 18:49:17 on dist-test-slave-fgb3"
server_key: "82b79bf141f70dd252ce8cc584f60f93"
server_key_iv: "3c2f5023528575815edd083c4c6b9646"
server_key_version: "encryptionkey@0"
I20260806 18:49:17.774485 1855 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:49:17.776993 1879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:17.778213 1855 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:49:17.778357 1855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "ac3f41d0c0a249dbbd308d29e79e51f1"
format_stamp: "Formatted at 2026-08-06 18:49:17 on dist-test-slave-fgb3"
server_key: "82b79bf141f70dd252ce8cc584f60f93"
server_key_iv: "3c2f5023528575815edd083c4c6b9646"
server_key_version: "encryptionkey@0"
I20260806 18:49:17.778478 1855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:17.810957 1855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:17.814015 1855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:17.814208 1855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:17.821305 1855 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:35511
I20260806 18:49:17.821343 1949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:35511 every 8 connection(s)
I20260806 18:49:17.822244 1855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:17.825064 1950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:17.830825 1950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1: Bootstrap starting.
I20260806 18:49:17.832516 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 1855
I20260806 18:49:17.832661 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:17.832957 2005 external_mini_cluster.cc:1428] Setting key a89db1db6bdd27f878e4a6efaedc25b9
I20260806 18:49:17.833637 1950 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:17.834517 1950 log.cc:824] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:17.836791 1950 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1: No bootstrap required, opened a new log
I20260806 18:49:17.841177 1950 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f41d0c0a249dbbd308d29e79e51f1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35511 } }
I20260806 18:49:17.841490 1950 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:17.841557 1950 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac3f41d0c0a249dbbd308d29e79e51f1, State: Initialized, Role: FOLLOWER
I20260806 18:49:17.842072 1950 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f41d0c0a249dbbd308d29e79e51f1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35511 } }
I20260806 18:49:17.842235 1950 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:17.842332 1950 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:17.842451 1950 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:17.843060 1953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:17.834234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43215 (local address 127.1.245.126:35511)
0806 18:49:17.834756 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:17.834777 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:17.834833 (+ 56us) server_negotiation.cc:407] Connection header received
0806 18:49:17.835593 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:17.835616 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:17.836040 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:17.836413 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:17.836852 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:17.838366 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:17.839258 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:17.840024 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:17.840184 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:17.840915 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:17.840921 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:17.840937 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:17.841436 (+ 499us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:17.841512 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:17.841546 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:17.841647 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:17.841770 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 18:49:17.841994 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":148,"threads_started":1}
I20260806 18:49:17.843632 1950 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f41d0c0a249dbbd308d29e79e51f1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35511 } }
I20260806 18:49:17.843966 1950 leader_election.cc:302] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac3f41d0c0a249dbbd308d29e79e51f1; no voters:
I20260806 18:49:17.844326 1950 leader_election.cc:288] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:17.844430 1956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:17.844669 1956 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [term 1 LEADER]: Becoming Leader. State: Replica: ac3f41d0c0a249dbbd308d29e79e51f1, State: Running, Role: LEADER
I20260806 18:49:17.845058 1956 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f41d0c0a249dbbd308d29e79e51f1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35511 } }
I20260806 18:49:17.845655 1950 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:17.846118 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:17.846868 1958 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac3f41d0c0a249dbbd308d29e79e51f1. Latest consensus state: current_term: 1 leader_uuid: "ac3f41d0c0a249dbbd308d29e79e51f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f41d0c0a249dbbd308d29e79e51f1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35511 } } }
I20260806 18:49:17.846971 1958 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:17.847208 1957 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac3f41d0c0a249dbbd308d29e79e51f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac3f41d0c0a249dbbd308d29e79e51f1" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35511 } } }
I20260806 18:49:17.847285 1957 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:17.847333 1972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:17.850494 1972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:17.853974 1953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:17.848581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38311 (local address 127.1.245.126:35511)
0806 18:49:17.848724 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:17.848728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:17.849250 (+ 522us) server_negotiation.cc:407] Connection header received
0806 18:49:17.849406 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:17.849409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:17.849481 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:17.849577 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:17.849896 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:17.851116 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:17.852241 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:17.852585 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:17.852697 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:17.853548 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:17.853552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:17.853555 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:17.853568 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:17.853599 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:17.853613 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:17.853690 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:17.853777 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 18:49:17.853842 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 18:49:17.856501 1972 catalog_manager.cc:1347] Generated new cluster ID: 0926eb60929e48c4be93484e16c38220
I20260806 18:49:17.856606 1972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:17.868669 1972 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:17.869587 1972 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:17.875449 1972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac3f41d0c0a249dbbd308d29e79e51f1: Generated new TSK 0
I20260806 18:49:17.876106 1972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:17.883288 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 1855
2026-08-06T18:49:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:17 dist-test-slave-fgb3 krb5kdc[1999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:17 dist-test-slave-fgb3 krb5kdc[1999](info): set up 2 sockets
Aug 06 18:49:17 dist-test-slave-fgb3 krb5kdc[1999](info): commencing operation
krb5kdc: starting...
W20260806 18:49:19.004240 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:19 dist-test-slave-fgb3 krb5kdc[1999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:49:19.128903 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35861
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:33779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:19.259567 2024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:19.259851 2024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:19.259895 2024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:19.263916 2024 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:19.263991 2024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:19.264012 2024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:19.264031 2024 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:19.264048 2024 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:19.269577 2024 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:33779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:19.270727 2024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:19.271797 2024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:19.280870 2031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:19.280877 2033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:19.280941 2024 server_base.cc:1034] running on GCE node
W20260806 18:49:19.280954 2030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:19.281684 2024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:19.282847 2024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:19.284015 2024 hybrid_clock.cc:648] HybridClock initialized: now 1786042159283994 us; error 34 us; skew 500 ppm
Aug 06 18:49:19 dist-test-slave-fgb3 krb5kdc[1999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042159, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:19.286768 2024 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:49:19.287804 2024 webserver.cc:460] Webserver started at http://127.1.245.126:42109/ using document root <none> and password file <none>
I20260806 18:49:19.288515 2024 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:19.288609 2024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:19.288883 2024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:19.291270 2024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "7df53903d75e4d6dbcc0eb11b3abd80b"
format_stamp: "Formatted at 2026-08-06 18:49:19 on dist-test-slave-fgb3"
server_key: "362842e72e105b5cf116e4f7776b199e"
server_key_iv: "eb9f3da78fe253cde4368dc9aed1c744"
server_key_version: "encryptionkey@0"
I20260806 18:49:19.291800 2024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "7df53903d75e4d6dbcc0eb11b3abd80b"
format_stamp: "Formatted at 2026-08-06 18:49:19 on dist-test-slave-fgb3"
server_key: "362842e72e105b5cf116e4f7776b199e"
server_key_iv: "eb9f3da78fe253cde4368dc9aed1c744"
server_key_version: "encryptionkey@0"
I20260806 18:49:19.295591 2024 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:49:19.298048 2043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:19.299340 2024 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:49:19.299456 2024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "7df53903d75e4d6dbcc0eb11b3abd80b"
format_stamp: "Formatted at 2026-08-06 18:49:19 on dist-test-slave-fgb3"
server_key: "362842e72e105b5cf116e4f7776b199e"
server_key_iv: "eb9f3da78fe253cde4368dc9aed1c744"
server_key_version: "encryptionkey@0"
I20260806 18:49:19.299556 2024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:19.304150 2024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:19.306972 2024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:19.307134 2024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:19.314703 2024 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:35861
I20260806 18:49:19.314711 2114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:35861 every 8 connection(s)
I20260806 18:49:19.315939 2024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:19.318964 2115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:19.326112 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2024
I20260806 18:49:19.326256 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:19.326354 2115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b: Bootstrap starting.
I20260806 18:49:19.326577 2005 external_mini_cluster.cc:1428] Setting key 1c0268cd043a7176db3ccedd5d4133b4
I20260806 18:49:19.328742 2115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:19.329499 2115 log.cc:824] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:19.331626 2115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b: No bootstrap required, opened a new log
I20260806 18:49:19.331902 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:19.327822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36703 (local address 127.1.245.126:35861)
0806 18:49:19.328297 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:19.328317 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:19.328481 (+ 164us) server_negotiation.cc:407] Connection header received
0806 18:49:19.329241 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:19.329261 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:19.329556 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:19.329889 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:19.330701 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:19.330706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:19.330718 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:19.330729 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:19.330782 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:19.330801 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:19.330897 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:19.331016 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 18:49:19.331176 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":128,"threads_started":1}
I20260806 18:49:19.334470 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:19.335234 2115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df53903d75e4d6dbcc0eb11b3abd80b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35861 } }
I20260806 18:49:19.335438 2115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 18:49:19.335472 2115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7df53903d75e4d6dbcc0eb11b3abd80b, State: Initialized, Role: FOLLOWER
I20260806 18:49:19.335894 2115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df53903d75e4d6dbcc0eb11b3abd80b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35861 } }
I20260806 18:49:19.336020 2115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 18:49:19.336078 2115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 18:49:19.336165 2115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:19.337225 2115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df53903d75e4d6dbcc0eb11b3abd80b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35861 } }
I20260806 18:49:19.337585 2115 leader_election.cc:302] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7df53903d75e4d6dbcc0eb11b3abd80b; no voters:
I20260806 18:49:19.337927 2118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:19.336328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43759 (local address 127.1.245.126:35861)
0806 18:49:19.336450 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:19.336454 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:19.336621 (+ 167us) server_negotiation.cc:407] Connection header received
0806 18:49:19.336705 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:19.336708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:19.336741 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:19.336821 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:19.337519 (+ 698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:19.337523 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:19.337526 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 18:49:19.337537 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:19.337566 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 18:49:19.337578 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:19.337666 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:19.337775 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 18:49:19.337814 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 18:49:19.338001 2115 leader_election.cc:288] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 18:49:19.338073 2132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:19.338340 2132 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [term 1 LEADER]: Becoming Leader. State: Replica: 7df53903d75e4d6dbcc0eb11b3abd80b, State: Running, Role: LEADER
I20260806 18:49:19.338712 2132 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df53903d75e4d6dbcc0eb11b3abd80b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35861 } }
I20260806 18:49:19.339175 2115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:19.339870 2132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7df53903d75e4d6dbcc0eb11b3abd80b. Latest consensus state: current_term: 1 leader_uuid: "7df53903d75e4d6dbcc0eb11b3abd80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df53903d75e4d6dbcc0eb11b3abd80b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35861 } } }
I20260806 18:49:19.339984 2132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:19.340287 2133 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7df53903d75e4d6dbcc0eb11b3abd80b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7df53903d75e4d6dbcc0eb11b3abd80b" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35861 } } }
I20260806 18:49:19.340432 2133 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:19.340495 2147 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:19.343556 2147 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:19.348752 2147 catalog_manager.cc:1347] Generated new cluster ID: 1008d0a48b1041009d90cac20d17f57a
I20260806 18:49:19.348809 2147 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:19.360195 2147 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:19.360481 2147 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:19.366735 2147 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7df53903d75e4d6dbcc0eb11b3abd80b: Generated new TSK 0
I20260806 18:49:19.367343 2147 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 18:49:19.372310 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2024
2026-08-06T18:49:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1491 ms)
[----------] 18 tests from AuthTokenIssuingTest (27046 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:19 dist-test-slave-fgb3 krb5kdc[2165](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:19 dist-test-slave-fgb3 krb5kdc[2165](info): set up 2 sockets
Aug 06 18:49:19 dist-test-slave-fgb3 krb5kdc[2165](info): commencing operation
krb5kdc: starting...
W20260806 18:49:20.489218 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:20 dist-test-slave-fgb3 krb5kdc[2165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:49:20.618515 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:38215
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:38215,127.1.245.125:35783,127.1.245.124:36313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:20.730978 2184 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:49:20.731293 2184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:20.731318 2184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:20.731379 2184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:20.735426 2184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:20.735487 2184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:20.735507 2184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:20.735522 2184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:20.735536 2184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:20.740939 2184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:38215,127.1.245.125:35783,127.1.245.124:36313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:38215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:20.742093 2184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:20.743242 2184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:20.751938 2191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:20.751935 2190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:20.752064 2184 server_base.cc:1034] running on GCE node
W20260806 18:49:20.751935 2194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:20.752753 2184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:20.753839 2184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:20.755005 2184 hybrid_clock.cc:648] HybridClock initialized: now 1786042160754988 us; error 30 us; skew 500 ppm
Aug 06 18:49:20 dist-test-slave-fgb3 krb5kdc[2165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:20.767633 2184 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:49:20.768697 2184 webserver.cc:460] Webserver started at http://127.1.245.126:44805/ using document root <none> and password file <none>
I20260806 18:49:20.769279 2184 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:20.769345 2184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:20.769520 2184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:20.771709 2184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "97b1d99a91d84e7aa54a1461663b7f9a"
format_stamp: "Formatted at 2026-08-06 18:49:20 on dist-test-slave-fgb3"
server_key: "2ab1b193194fa2e74a90b7993320bd77"
server_key_iv: "e7cd286162e620c25bbf06140ddb5899"
server_key_version: "encryptionkey@0"
I20260806 18:49:20.772228 2184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "97b1d99a91d84e7aa54a1461663b7f9a"
format_stamp: "Formatted at 2026-08-06 18:49:20 on dist-test-slave-fgb3"
server_key: "2ab1b193194fa2e74a90b7993320bd77"
server_key_iv: "e7cd286162e620c25bbf06140ddb5899"
server_key_version: "encryptionkey@0"
I20260806 18:49:20.776115 2184 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260806 18:49:20.778748 2204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:20.780042 2184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260806 18:49:20.780153 2184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "97b1d99a91d84e7aa54a1461663b7f9a"
format_stamp: "Formatted at 2026-08-06 18:49:20 on dist-test-slave-fgb3"
server_key: "2ab1b193194fa2e74a90b7993320bd77"
server_key_iv: "e7cd286162e620c25bbf06140ddb5899"
server_key_version: "encryptionkey@0"
I20260806 18:49:20.780246 2184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:20.791707 2184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:20.794842 2184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:20.795010 2184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:20.802242 2184 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:38215
I20260806 18:49:20.802531 2273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:38215 every 8 connection(s)
I20260806 18:49:20.803248 2184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:20.805218 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2184
I20260806 18:49:20.805348 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:20.805660 2005 external_mini_cluster.cc:1428] Setting key 009b9bb9336588cd60ba9db3190a975d
I20260806 18:49:20.806433 2274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:20.811129 2274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.245.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.125@KRBTEST.COM" created.
Aug 06 18:49:20 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:20.826861 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:20.813929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44428 (local address 127.1.245.126:38215)
0806 18:49:20.814518 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:20.814535 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:20.814699 (+ 164us) server_negotiation.cc:407] Connection header received
0806 18:49:20.815823 (+ 1124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:20.815828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:20.816052 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:20.816200 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:20.816985 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:20.817954 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:20.818970 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:20.819556 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:20.819683 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:20.822162 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:20.822168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:20.822179 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:20.822201 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:20.823729 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:20.824172 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:20.824177 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:20.824179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:20.824283 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:20.824572 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:20.824575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:20.824577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:20.824959 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:20.825081 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:20.825448 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 18:49:20.825747 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":463,"spinlock_wait_cycles":29568,"thread_start_us":151,"threads_started":1}
I20260806 18:49:20.826864 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:20.813929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38215 (local address 127.0.0.1:44428)
0806 18:49:20.814536 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:20.814588 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:20.815514 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:20.816245 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:20.816249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:20.816593 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:20.816864 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:20.816878 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:20.818074 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:20.818078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:20.818847 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:20.818857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:20.819729 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:20.819733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:20.819856 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:20.820457 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:20.820479 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:20.822020 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:20.823837 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:20.823846 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:20.823858 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:20.824071 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:20.824375 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:20.824378 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:20.824380 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:20.824490 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:20.825067 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:20.825073 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:20.825215 (+ 142us) client_negotiation.cc:769] Sending connection context
0806 18:49:20.825370 (+ 155us) client_negotiation.cc:240] Negotiation successful
0806 18:49:20.825810 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":131,"thread_start_us":149,"threads_started":1}
I20260806 18:49:20.829420 2274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:20.830360 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:20.829950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52512)
0806 18:49:20.830093 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:20.830200 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260806 18:49:20.830551 2205 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
I20260806 18:49:20.831454 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:20.831231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52528)
0806 18:49:20.831346 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:20.831394 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:49:20.831593 2274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:35783: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab.
Entry for principal kudu/127.1.245.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab.
I20260806 18:49:20.833828 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.125:35783
--webserver_interface=127.1.245.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab
--principal=kudu/127.1.245.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:38215,127.1.245.125:35783,127.1.245.124:36313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:49:20.880887 2274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } attempt: 1
I20260806 18:49:20.881948 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:20.881591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52538)
0806 18:49:20.881792 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:20.881862 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260806 18:49:20.882694 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:20.882493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52544)
0806 18:49:20.882585 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:20.882632 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260806 18:49:20.882838 2274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:35783: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
W20260806 18:49:20.947443 2284 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:49:20.947757 2284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:20.947783 2284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:20.947835 2284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:49:20.951005 2274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } attempt: 2
I20260806 18:49:20.951903 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:20.951588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52556)
0806 18:49:20.951750 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:20.951817 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260806 18:49:20.952224 2284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:20.952301 2284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:20.952328 2284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:20.952360 2284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:20.952380 2284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:20.952627 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:20.952401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52564)
0806 18:49:20.952516 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:20.952565 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260806 18:49:20.952775 2274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:35783: Network error: Client connection negotiation failed: client connection to 127.1.245.125:35783: connect: Connection refused (error 111)
I20260806 18:49:20.958079 2284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:38215,127.1.245.125:35783,127.1.245.124:36313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab
--principal=kudu/127.1.245.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.125:35783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.245.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:20.959286 2284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:20.960347 2284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:20.969318 2290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:20.969357 2291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:20.969441 2294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:20.969522 2284 server_base.cc:1034] running on GCE node
I20260806 18:49:20.970252 2284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:20.971495 2284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:20.972678 2284 hybrid_clock.cc:648] HybridClock initialized: now 1786042160972657 us; error 32 us; skew 500 ppm
Aug 06 18:49:20 dist-test-slave-fgb3 krb5kdc[2165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:20.975498 2284 init.cc:377] Logged in from keytab as kudu/127.1.245.125@KRBTEST.COM (short username kudu)
I20260806 18:49:20.976397 2284 webserver.cc:460] Webserver started at http://127.1.245.125:42993/ using document root <none> and password file <none>
I20260806 18:49:20.976951 2284 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:20.977021 2284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:20.977226 2284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:20.979326 2284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/data/instance:
uuid: "77ff31328f0e4e318961ab19045ce23b"
format_stamp: "Formatted at 2026-08-06 18:49:20 on dist-test-slave-fgb3"
server_key: "7dce8b502cbd1bc088998854d0ef1dc9"
server_key_iv: "77ef30fd73317647b1d5ee757838831e"
server_key_version: "encryptionkey@0"
I20260806 18:49:20.979868 2284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/wal/instance:
uuid: "77ff31328f0e4e318961ab19045ce23b"
format_stamp: "Formatted at 2026-08-06 18:49:20 on dist-test-slave-fgb3"
server_key: "7dce8b502cbd1bc088998854d0ef1dc9"
server_key_iv: "77ef30fd73317647b1d5ee757838831e"
server_key_version: "encryptionkey@0"
I20260806 18:49:20.983736 2284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:49:20.986114 2302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:20.987330 2284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:49:20.987455 2284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/wal
uuid: "77ff31328f0e4e318961ab19045ce23b"
format_stamp: "Formatted at 2026-08-06 18:49:20 on dist-test-slave-fgb3"
server_key: "7dce8b502cbd1bc088998854d0ef1dc9"
server_key_iv: "77ef30fd73317647b1d5ee757838831e"
server_key_version: "encryptionkey@0"
I20260806 18:49:20.987547 2284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:21.026532 2284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:21.029776 2284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:21.029976 2284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:21.036957 2284 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.125:35783
I20260806 18:49:21.037269 2373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.125:35783 every 8 connection(s)
I20260806 18:49:21.037922 2284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
I20260806 18:49:21.040746 2374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:21.041332 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2284
I20260806 18:49:21.041460 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-1/wal/instance
I20260806 18:49:21.041772 2005 external_mini_cluster.cc:1428] Setting key 57e4a17a069731eaa2b3a27efac537e3
I20260806 18:49:21.043944 2274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } attempt: 3
I20260806 18:49:21.045497 2374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.245.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.124@KRBTEST.COM" created.
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:21.058957 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.044514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52578)
0806 18:49:21.044744 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.044761 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:21.044863 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:21.047918 (+ 3055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:21.047922 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:21.048127 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:21.048284 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.048292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.050465 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.050468 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.051131 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.051141 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.051845 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.051849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.051951 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.052348 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:21.052362 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:21.054670 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:21.057118 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.057120 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.057122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.057355 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.057671 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.057674 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.057676 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.057755 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.058413 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:21.058416 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:21.058523 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 18:49:21.058630 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:49:21.058763 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"spinlock_wait_cycles":896}
I20260806 18:49:21.060298 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.047385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44442 (local address 127.1.245.126:38215)
0806 18:49:21.047501 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.047505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.047856 (+ 351us) server_negotiation.cc:407] Connection header received
0806 18:49:21.048040 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.048043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.048094 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.048137 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.049170 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.049679 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.050955 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.051311 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.051427 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.055462 (+ 4035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.055465 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.055467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.055489 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.057729 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.058843 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.058846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.058848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.058928 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.059247 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.059250 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.059252 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.059471 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.059569 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.060031 (+ 462us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.060151 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46,"spinlock_wait_cycles":10368}
I20260806 18:49:21.060923 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.045545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52578 (local address 127.1.245.125:35783)
0806 18:49:21.046050 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.046070 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.046115 (+ 45us) server_negotiation.cc:407] Connection header received
0806 18:49:21.047091 (+ 976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.047119 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.047579 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.048476 (+ 897us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.048964 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.050300 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.051244 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.051695 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.051776 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.054840 (+ 3064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.054848 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.054862 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.054888 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.056841 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.057465 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.057470 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.057476 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.057582 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.057837 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.057840 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.057842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.058264 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.058458 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.059500 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.059861 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":328,"thread_start_us":139,"threads_started":1}
I20260806 18:49:21.060932 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.047352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38215 (local address 127.0.0.1:44442)
0806 18:49:21.047732 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.047781 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:21.047939 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:21.048352 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:21.048359 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:21.048664 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:21.049037 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.049048 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.049787 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.049790 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.050784 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.050794 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.051492 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.051496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.051648 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.052316 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:21.052337 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:21.055304 (+ 2967us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:21.058405 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.058415 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.058422 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.058718 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.059014 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.059018 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.059020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.059114 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.059588 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:21.059596 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:21.059778 (+ 182us) client_negotiation.cc:769] Sending connection context
0806 18:49:21.059936 (+ 158us) client_negotiation.cc:240] Negotiation successful
0806 18:49:21.060137 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":71,"thread_start_us":158,"threads_started":1}
I20260806 18:49:21.062216 2374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:21.063199 2274 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:21.063928 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.063631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41638)
0806 18:49:21.063786 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.063844 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260806 18:49:21.064679 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.064426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41652)
0806 18:49:21.064585 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.064629 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260806 18:49:21.064792 2274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36313: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab.
Entry for principal kudu/127.1.245.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab.
I20260806 18:49:21.070945 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.063155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52594)
0806 18:49:21.063276 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.063288 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:21.063357 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:21.063583 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:21.063585 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:21.063741 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:21.063845 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.063852 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.064836 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.064840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.065549 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.065559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.065986 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.065989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.066113 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.066681 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:21.066697 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:21.067967 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:21.069600 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.069603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.069604 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.069813 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.070052 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.070054 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.070055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.070120 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.070516 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:21.070518 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:21.070596 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 18:49:21.070671 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 18:49:21.070791 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":20}
I20260806 18:49:21.071053 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.063297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52594 (local address 127.1.245.125:35783)
0806 18:49:21.063401 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.063406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.063417 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:49:21.063444 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.063447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.063483 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.063558 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.063979 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.064645 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.065656 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.065892 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.065977 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.068106 (+ 2129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.068110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.068112 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.068155 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.069499 (+ 1344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.069914 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.069917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.069918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.069990 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.070199 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.070202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.070204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.070435 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.070512 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.070743 (+ 231us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.070854 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":46}
I20260806 18:49:21.071615 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.124:36313
--webserver_interface=127.1.245.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab
--principal=kudu/127.1.245.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:42669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:38215,127.1.245.125:35783,127.1.245.124:36313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:49:21.071691 2374 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:21.072341 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.072100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41658)
0806 18:49:21.072207 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.072291 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 18:49:21.072546 2309 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
I20260806 18:49:21.073403 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.073204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41662)
0806 18:49:21.073303 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.073346 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260806 18:49:21.073534 2374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36313: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
I20260806 18:49:21.095938 2274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } attempt: 1
I20260806 18:49:21.096835 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.096509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41666)
0806 18:49:21.096681 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.096746 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260806 18:49:21.097544 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.097353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41674)
0806 18:49:21.097442 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.097487 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 18:49:21.097664 2274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36313: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
I20260806 18:49:21.122797 2374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } attempt: 1
I20260806 18:49:21.123698 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.123385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41690)
0806 18:49:21.123550 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.123614 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 18:49:21.124383 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.124174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41704)
0806 18:49:21.124276 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.124321 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:49:21.124526 2374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36313: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
I20260806 18:49:21.172806 2274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } attempt: 2
I20260806 18:49:21.173687 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.173390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41720)
0806 18:49:21.173539 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.173603 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:49:21.174440 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.174193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41732)
0806 18:49:21.174298 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.174376 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 18:49:21.174562 2274 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36313: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
W20260806 18:49:21.184729 2382 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:49:21.185030 2382 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:21.185077 2382 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:21.185140 2382 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:21.189204 2382 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:21.189270 2382 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:21.189296 2382 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:21.189319 2382 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:21.189347 2382 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:21.192685 2374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } attempt: 2
I20260806 18:49:21.193537 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.193174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41738)
0806 18:49:21.193352 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.193437 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 18:49:21.194682 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.194431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41740)
0806 18:49:21.194575 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.194621 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260806 18:49:21.194839 2374 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36313: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36313: connect: Connection refused (error 111)
I20260806 18:49:21.195317 2382 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:42669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:38215,127.1.245.125:35783,127.1.245.124:36313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab
--principal=kudu/127.1.245.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.124:36313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.245.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2382
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:21.196403 2382 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:21.197458 2382 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 18:49:21.206050 2382 server_base.cc:1034] running on GCE node
W20260806 18:49:21.206007 2391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:21.206043 2395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:21.206007 2398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:21.207000 2382 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:21.208163 2382 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:21.209360 2382 hybrid_clock.cc:648] HybridClock initialized: now 1786042161209330 us; error 41 us; skew 500 ppm
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042161, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:21.212438 2382 init.cc:377] Logged in from keytab as kudu/127.1.245.124@KRBTEST.COM (short username kudu)
I20260806 18:49:21.213479 2382 webserver.cc:460] Webserver started at http://127.1.245.124:39821/ using document root <none> and password file <none>
I20260806 18:49:21.214083 2382 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:21.214160 2382 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:21.214442 2382 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:21.216655 2382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/data/instance:
uuid: "c403b70667574e5a8c417acdf21c6d5b"
format_stamp: "Formatted at 2026-08-06 18:49:21 on dist-test-slave-fgb3"
server_key: "cb9003c276dd2a8e9cf116c2fbef6f09"
server_key_iv: "e09ded36ee7dc030f892eb5d35f6caf8"
server_key_version: "encryptionkey@0"
I20260806 18:49:21.217208 2382 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/wal/instance:
uuid: "c403b70667574e5a8c417acdf21c6d5b"
format_stamp: "Formatted at 2026-08-06 18:49:21 on dist-test-slave-fgb3"
server_key: "cb9003c276dd2a8e9cf116c2fbef6f09"
server_key_iv: "e09ded36ee7dc030f892eb5d35f6caf8"
server_key_version: "encryptionkey@0"
I20260806 18:49:21.221089 2382 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260806 18:49:21.223773 2410 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:21.225111 2382 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260806 18:49:21.225270 2382 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/wal
uuid: "c403b70667574e5a8c417acdf21c6d5b"
format_stamp: "Formatted at 2026-08-06 18:49:21 on dist-test-slave-fgb3"
server_key: "cb9003c276dd2a8e9cf116c2fbef6f09"
server_key_iv: "e09ded36ee7dc030f892eb5d35f6caf8"
server_key_version: "encryptionkey@0"
I20260806 18:49:21.225379 2382 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:21.248503 2382 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:21.251695 2382 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:21.251892 2382 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:21.259087 2382 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.124:36313
I20260806 18:49:21.259366 2488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.124:36313 every 8 connection(s)
I20260806 18:49:21.260056 2382 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
I20260806 18:49:21.262948 2489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:21.266844 2489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:21.268797 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2382
I20260806 18:49:21.268934 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786041874296929-2005-0/minicluster-data/master-2/wal/instance
I20260806 18:49:21.269270 2005 external_mini_cluster.cc:1428] Setting key e1ba29e85cf700a4b6db3ce8d1c54523
I20260806 18:49:21.273723 2274 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } attempt: 3
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042161, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
I20260806 18:49:21.282459 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.268848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44452 (local address 127.1.245.126:38215)
0806 18:49:21.269020 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.269026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.269935 (+ 909us) server_negotiation.cc:407] Connection header received
0806 18:49:21.271118 (+ 1183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.271122 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.271175 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.271263 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.272119 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.272626 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.273857 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.274211 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.274351 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.277459 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.277463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.277465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.277492 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.279137 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.279778 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.279781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.279783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.279876 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.281303 (+ 1427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.281307 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.281309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.281538 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.281619 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.282127 (+ 508us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.282280 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260806 18:49:21.283515 2493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.269980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41744 (local address 127.1.245.124:36313)
0806 18:49:21.270525 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.270534 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.270569 (+ 35us) server_negotiation.cc:407] Connection header received
0806 18:49:21.270895 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.270914 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.271131 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.271311 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.272820 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.273705 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.274750 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.275198 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.275298 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.277841 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.277848 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.277863 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.277883 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.279698 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.280519 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.280526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.280528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.280674 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.280948 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.280968 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.280970 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.281497 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.281697 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.281936 (+ 239us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.282258 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":448,"spinlock_wait_cycles":2944,"thread_start_us":158,"threads_started":1}
I20260806 18:49:21.283687 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.274171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41752)
0806 18:49:21.274438 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.274451 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:21.274529 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:21.274985 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:21.274988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:21.275497 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:21.275578 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.275583 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.276435 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.276438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.276990 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.277000 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.277459 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.277463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.277544 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.278138 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:21.278151 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:21.279533 (+ 1382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:21.282112 (+ 2579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.282115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.282117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.282441 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.282725 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.282729 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.282731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.282811 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.283176 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:21.283179 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:21.283272 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 18:49:21.283350 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 18:49:21.283485 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260806 18:49:21.283515 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.269191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:38215 (local address 127.0.0.1:44452)
0806 18:49:21.269756 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.269811 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:21.270890 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:21.271351 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:21.271356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:21.271689 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:21.271966 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.271980 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.272769 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.272773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.273738 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.273751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.274514 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.274518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.274733 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.275411 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:21.275432 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:21.277276 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:21.279237 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.279246 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.279257 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.279681 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.281076 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.281080 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.281083 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.281192 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.281669 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:21.281676 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:21.281830 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 18:49:21.282603 (+ 773us) client_negotiation.cc:240] Negotiation successful
0806 18:49:21.282799 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":40,"thread_start_us":154,"threads_started":1}
I20260806 18:49:21.283733 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.274194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.1.245.124:36313)
0806 18:49:21.274753 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.274765 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.274778 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:49:21.274809 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.274812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.274856 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.274945 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.275685 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.276330 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.277094 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.277332 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.277747 (+ 415us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.279682 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.279685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.279687 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.279708 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.281992 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.282529 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.282532 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.282534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.282618 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.282908 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.282911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.282913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.283095 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.283186 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.283428 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.283583 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":146,"threads_started":1}
I20260806 18:49:21.284921 2489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:21.285972 2374 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } attempt: 3
I20260806 18:49:21.288470 2274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a: Bootstrap starting.
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042161, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
I20260806 18:49:21.291407 2274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:21.292383 2274 log.cc:824] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:21.294034 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.285447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:35783 (local address 127.0.0.1:52596)
0806 18:49:21.285711 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.285725 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:21.285804 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:21.286142 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:21.286145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:21.286306 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:21.286563 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.286570 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.287409 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.287411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.287924 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.287930 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.288359 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.288362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.288467 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.288941 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:21.288955 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:21.290290 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:21.292280 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.292283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.292286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.292518 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.292848 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.292851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.292853 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.292940 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.293512 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:21.293515 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:21.293617 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 18:49:21.293727 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 18:49:21.293862 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":1}
I20260806 18:49:21.294080 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.285718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52596 (local address 127.1.245.125:35783)
0806 18:49:21.285845 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.285849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.285863 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:49:21.285905 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.285908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.285953 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.286025 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.286695 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.287294 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.288027 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.288262 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.288346 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.290488 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.290492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.290495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.290522 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.292137 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.292634 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.292637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.292639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.292730 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.293042 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.293045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.293047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.293361 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.293438 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.293809 (+ 371us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.293923 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 18:49:21.295529 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.286386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41762)
0806 18:49:21.286560 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.286573 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:21.286644 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:21.287105 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:21.287109 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:21.287294 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:21.287392 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.287399 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.288445 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.288448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.289169 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.289181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.289775 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.289779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.289863 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.290349 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:21.290364 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:21.291827 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:21.293586 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.293589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.293591 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.293804 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.294116 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.294119 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.294121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.294611 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.295074 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:21.295077 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:21.295170 (+ 93us) client_negotiation.cc:769] Sending connection context
0806 18:49:21.295256 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 18:49:21.295384 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260806 18:49:21.296145 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.286809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41762 (local address 127.1.245.124:36313)
0806 18:49:21.286894 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.286898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.286909 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:49:21.286938 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.286941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.286979 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.287501 (+ 522us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.287583 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.288315 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.289325 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.289647 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.289754 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.292062 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.292065 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.292067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.292088 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.293489 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.293924 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.293927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.293928 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.294008 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.294739 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.294742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.294743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.294964 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.295770 (+ 806us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.295867 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.295999 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":34}
I20260806 18:49:21.297073 2489 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:21.297806 2274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a: No bootstrap required, opened a new log
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:21.300805 2374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b: Bootstrap starting.
I20260806 18:49:21.302490 2274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } }
I20260806 18:49:21.302726 2274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97b1d99a91d84e7aa54a1461663b7f9a, State: Initialized, Role: FOLLOWER
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042161, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
I20260806 18:49:21.303081 2274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } }
I20260806 18:49:21.303824 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.292935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44464 (local address 127.1.245.126:38215)
0806 18:49:21.293057 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.293061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.293074 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:49:21.293112 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.293114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.293157 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.293230 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.294150 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.294922 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.295984 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.296414 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.296552 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.299897 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.299900 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.299903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.299932 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.301581 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.302085 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.302088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.302090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.302179 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.302515 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.302519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.302520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.302739 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.302905 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.303564 (+ 659us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.303679 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260806 18:49:21.303936 2374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:21.304891 2374 log.cc:824] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:21.304967 2274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:21.306581 2491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.297792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36313 (local address 127.0.0.1:41772)
0806 18:49:21.298014 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:21.298027 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:21.298095 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:21.298705 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:21.298708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:21.298860 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:21.298956 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.298962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.299848 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.299851 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.300597 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:21.300609 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.301206 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.301209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:21.301278 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.301730 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:21.301744 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:21.303240 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:21.305160 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.305162 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.305164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.305414 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.305650 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:21.305653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:21.305654 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:21.305717 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:21.306093 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:21.306096 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:21.306178 (+ 82us) client_negotiation.cc:769] Sending connection context
0806 18:49:21.306250 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 18:49:21.306418 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260806 18:49:21.307142 2497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.307940 2497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:21.308763 2374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b: No bootstrap required, opened a new log
I20260806 18:49:21.311885 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.297984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.1.245.124:36313)
0806 18:49:21.298452 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.298456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.298469 (+ 13us) server_negotiation.cc:407] Connection header received
0806 18:49:21.298500 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.298502 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.298540 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.298608 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.299062 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.299740 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.300717 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.301010 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.301100 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.303362 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.303366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.303368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.303389 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.305051 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.305511 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.305514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.305515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.305582 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.305786 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.305789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.305791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.305991 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.311491 (+ 5500us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.311593 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.311726 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398}
Aug 06 18:49:21 dist-test-slave-fgb3 krb5kdc[2165](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
I20260806 18:49:21.310217 2184 master_runner.cc:186] Error getting master registration for 127.1.245.125:35783: OK, instance_id { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" instance_seqno: 1786042161035026 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:49:21.314190 2184 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:49:21.315436 2489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b: Bootstrap starting.
W20260806 18:49:21.315819 2512 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:21.315986 2512 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:21.318629 2489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:21.319494 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.305107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52606 (local address 127.1.245.125:35783)
0806 18:49:21.305219 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.305223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.305402 (+ 179us) server_negotiation.cc:407] Connection header received
0806 18:49:21.305580 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.305583 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.305630 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.305679 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.306565 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.307307 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.309738 (+ 2431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.310038 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.310112 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.313967 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.313971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.313975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.314031 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.317474 (+ 3443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.318046 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.318050 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.318052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.318152 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.318598 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.318602 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.318604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.318841 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.319087 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.319220 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.319338 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:49:21.319615 2489 log.cc:824] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:21.321904 2374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } }
I20260806 18:49:21.322227 2374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77ff31328f0e4e318961ab19045ce23b, State: Initialized, Role: FOLLOWER
I20260806 18:49:21.322729 2489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b: No bootstrap required, opened a new log
I20260806 18:49:21.322768 2374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } }
I20260806 18:49:21.322916 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 18:49:21.324234 2240 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "97b1d99a91d84e7aa54a1461663b7f9a"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44464
I20260806 18:49:21.324366 2374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:21.324620 2240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:49:21.324755 2240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:21.326102 2240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } }
I20260806 18:49:21.326762 2489 raft_consensus.cc:348] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } }
I20260806 18:49:21.327101 2489 raft_consensus.cc:729] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c403b70667574e5a8c417acdf21c6d5b, State: Initialized, Role: FOLLOWER
I20260806 18:49:21.327224 2240 leader_election.cc:288] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [CANDIDATE]: Term 1 election: Requested vote from peers 77ff31328f0e4e318961ab19045ce23b (127.1.245.125:35783), c403b70667574e5a8c417acdf21c6d5b (127.1.245.124:36313)
I20260806 18:49:21.327953 2339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "77ff31328f0e4e318961ab19045ce23b"
I20260806 18:49:21.328171 2339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:21.329456 2339 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b1d99a91d84e7aa54a1461663b7f9a in term 1.
I20260806 18:49:21.327493 2489 consensus_queue.cc:260] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } }
I20260806 18:49:21.330283 2205 leader_election.cc:302] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77ff31328f0e4e318961ab19045ce23b, 97b1d99a91d84e7aa54a1461663b7f9a; no voters:
I20260806 18:49:21.330621 2497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:21.330832 2497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [term 1 LEADER]: Becoming Leader. State: Replica: 97b1d99a91d84e7aa54a1461663b7f9a, State: Running, Role: LEADER
I20260806 18:49:21.331146 2497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } }
I20260806 18:49:21.331804 2489 sys_catalog.cc:564] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:21.332892 2497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97b1d99a91d84e7aa54a1461663b7f9a. Latest consensus state: current_term: 1 leader_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.333003 2497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:21.334653 2455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c403b70667574e5a8c417acdf21c6d5b"
I20260806 18:49:21.334894 2455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:21.336270 2455 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97b1d99a91d84e7aa54a1461663b7f9a in term 1.
W20260806 18:49:21.336688 2532 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:21.336771 2532 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:21.337059 2551 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260806 18:49:21.337373 2561 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:21.337489 2561 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:21.337726 2540 sys_catalog.cc:455] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.337893 2540 sys_catalog.cc:458] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:21.337965 2551 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:21.341804 2516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.331918 2284 master_runner.cc:186] Error getting master registration for 127.1.245.124:36313: OK, instance_id { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" instance_seqno: 1786042161257065 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:49:21.341952 2516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:21.342026 2284 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:49:21.345376 2455 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [term 1 FOLLOWER]: Refusing update from remote peer 97b1d99a91d84e7aa54a1461663b7f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:49:21.346011 2541 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:49:21.350564 2540 sys_catalog.cc:455] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97b1d99a91d84e7aa54a1461663b7f9a. Latest consensus state: current_term: 1 leader_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.350694 2540 sys_catalog.cc:458] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:21.351627 2339 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [term 1 FOLLOWER]: Refusing update from remote peer 97b1d99a91d84e7aa54a1461663b7f9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:49:21.352181 2497 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:49:21.355775 2516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97b1d99a91d84e7aa54a1461663b7f9a. Latest consensus state: current_term: 1 leader_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.355913 2516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:21.365231 2541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.365386 2541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:21.367017 2516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.367154 2516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:21.367481 2566 mvcc.cc:205] Tried to move back new op lower bound from 7315628692863406080 to 7315628692815749120. Current Snapshot: MvccSnapshot[applied={T|T < 7315628692863406080}]
I20260806 18:49:21.368259 2497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.368373 2497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:21.369542 2540 sys_catalog.cc:455] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "97b1d99a91d84e7aa54a1461663b7f9a" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 38215 } } peers { permanent_uuid: "77ff31328f0e4e318961ab19045ce23b" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 35783 } } peers { permanent_uuid: "c403b70667574e5a8c417acdf21c6d5b" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36313 } } }
I20260806 18:49:21.369668 2540 sys_catalog.cc:458] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:21.372493 2569 mvcc.cc:205] Tried to move back new op lower bound from 7315628692863406080 to 7315628692815749120. Current Snapshot: MvccSnapshot[applied={T|T < 7315628692863406080}]
I20260806 18:49:21.372690 2567 mvcc.cc:205] Tried to move back new op lower bound from 7315628692863406080 to 7315628692815749120. Current Snapshot: MvccSnapshot[applied={T|T < 7315628692863406080}]
I20260806 18:49:21.380179 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.357047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52618 (local address 127.1.245.125:35783)
0806 18:49:21.357151 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.357155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.371006 (+ 13851us) server_negotiation.cc:407] Connection header received
0806 18:49:21.371455 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.371459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.371505 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.371550 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.372883 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.373582 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.374645 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.374939 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.375014 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.376582 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.376586 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.376589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.376615 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.378246 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.378732 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.378736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.378737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.378825 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.379149 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.379153 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.379155 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.379399 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.379517 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.379894 (+ 377us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.380011 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260806 18:49:21.380686 2551 catalog_manager.cc:1347] Generated new cluster ID: 338d2ad27f5e4a4f81b54c6dd704849d
I20260806 18:49:21.380776 2551 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:21.395483 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.356410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44470 (local address 127.1.245.126:38215)
0806 18:49:21.356539 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.356544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.366600 (+ 10056us) server_negotiation.cc:407] Connection header received
0806 18:49:21.366825 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.366829 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.366883 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.366935 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.367994 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.368679 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.371289 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.371597 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.371667 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.378564 (+ 6897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.378568 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.378571 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.378602 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.380288 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.386655 (+ 6367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.386659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.386661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.386778 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.394450 (+ 7672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.394454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.394456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.394684 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.394826 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.395175 (+ 349us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.395311 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:49:21.406452 2551 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:21.407779 2551 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:21.417783 2494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:21.357865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41788 (local address 127.1.245.124:36313)
0806 18:49:21.358000 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.358004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.371244 (+ 13240us) server_negotiation.cc:407] Connection header received
0806 18:49:21.371813 (+ 569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.371816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.371864 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.371940 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.376320 (+ 4380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.377027 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.387050 (+ 10023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:21.387368 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:21.387468 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:21.413841 (+ 26373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:21.413846 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:21.413849 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:21.413888 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:21.415730 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.416312 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.416316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.416318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.416414 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:21.416764 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:21.416768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:21.416770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:21.417000 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:21.417110 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:21.417475 (+ 365us) server_negotiation.cc:299] Negotiation successful
0806 18:49:21.417605 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 18:49:21.442852 2551 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97b1d99a91d84e7aa54a1461663b7f9a: Generated new TSK 0
I20260806 18:49:21.443814 2551 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:49:21.471407 2494 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.469297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41798 (local address 127.1.245.124:36313)
0806 18:49:21.469445 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.469450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.470213 (+ 763us) server_negotiation.cc:407] Connection header received
0806 18:49:21.470459 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.470463 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.470518 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.470569 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.471256 (+ 687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41798: BlockingRecv error: recv got EOF from 127.0.0.1:41798 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260806 18:49:21.473943 2281 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.472142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44486 (local address 127.1.245.126:38215)
0806 18:49:21.472259 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.472263 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.472983 (+ 720us) server_negotiation.cc:407] Connection header received
0806 18:49:21.473165 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.473169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.473222 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.473275 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.473806 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44486: BlockingRecv error: recv got EOF from 127.0.0.1:44486 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:49:21.475561 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.472684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52626 (local address 127.1.245.125:35783)
0806 18:49:21.472814 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.472818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.474545 (+ 1727us) server_negotiation.cc:407] Connection header received
0806 18:49:21.474835 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.474838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.474889 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.474935 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.475435 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52626: BlockingRecv error: recv got EOF from 127.0.0.1:52626 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260806 18:49:21.485028 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.482805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52642 (local address 127.1.245.125:35783)
0806 18:49:21.482928 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.482933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.483670 (+ 737us) server_negotiation.cc:407] Connection header received
0806 18:49:21.483994 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.483999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.484092 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.484154 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.484906 (+ 752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52642: BlockingRecv error: recv got EOF from 127.0.0.1:52642 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 18:49:21.490547 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.488879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52658 (local address 127.1.245.125:35783)
0806 18:49:21.488985 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.488989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.489492 (+ 503us) server_negotiation.cc:407] Connection header received
0806 18:49:21.489696 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.489712 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.489769 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.489823 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.490430 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52658: BlockingRecv error: recv got EOF from 127.0.0.1:52658 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:49:21.498131 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.496267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52672 (local address 127.1.245.125:35783)
0806 18:49:21.496376 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.496380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.497037 (+ 657us) server_negotiation.cc:407] Connection header received
0806 18:49:21.497268 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.497272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.497332 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.497390 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.498007 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52672: BlockingRecv error: recv got EOF from 127.0.0.1:52672 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:49:21.506384 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.504651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52676 (local address 127.1.245.125:35783)
0806 18:49:21.504801 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.504806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.505378 (+ 572us) server_negotiation.cc:407] Connection header received
0806 18:49:21.505565 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.505568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.505618 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.505664 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.506197 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52676: BlockingRecv error: recv got EOF from 127.0.0.1:52676 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260806 18:49:21.518756 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.517089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.1.245.125:35783)
0806 18:49:21.517209 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.517214 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.517740 (+ 526us) server_negotiation.cc:407] Connection header received
0806 18:49:21.517938 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.517956 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.518011 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.518062 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.518618 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52684: BlockingRecv error: recv got EOF from 127.0.0.1:52684 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260806 18:49:21.538677 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.537348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.1.245.125:35783)
0806 18:49:21.537477 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.537482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.537744 (+ 262us) server_negotiation.cc:407] Connection header received
0806 18:49:21.537896 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.537899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.537960 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.538031 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.538545 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52696: BlockingRecv error: recv got EOF from 127.0.0.1:52696 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260806 18:49:21.573710 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.572622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52710 (local address 127.1.245.125:35783)
0806 18:49:21.572800 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.572804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.572934 (+ 130us) server_negotiation.cc:407] Connection header received
0806 18:49:21.573034 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.573037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.573093 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.573188 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.573601 (+ 413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52710: BlockingRecv error: recv got EOF from 127.0.0.1:52710 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260806 18:49:21.641121 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.639959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.1.245.125:35783)
0806 18:49:21.640119 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.640124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.640233 (+ 109us) server_negotiation.cc:407] Connection header received
0806 18:49:21.640396 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.640400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.640450 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.640520 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.640991 (+ 471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52712: BlockingRecv error: recv got EOF from 127.0.0.1:52712 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260806 18:49:21.772523 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:21.771367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52718 (local address 127.1.245.125:35783)
0806 18:49:21.771513 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:21.771517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:21.771651 (+ 134us) server_negotiation.cc:407] Connection header received
0806 18:49:21.771782 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:21.771785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:21.771838 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:21.771920 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:21.772415 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52718: BlockingRecv error: recv got EOF from 127.0.0.1:52718 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 18:49:22.031994 2378 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:22.030871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52734 (local address 127.1.245.125:35783)
0806 18:49:22.031046 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:22.031050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:22.031139 (+ 89us) server_negotiation.cc:407] Connection header received
0806 18:49:22.031266 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:22.031269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:22.031319 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:22.031433 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:22.031865 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52734: BlockingRecv error: recv got EOF from 127.0.0.1:52734 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260806 18:49:22.338222 2532 catalog_manager.cc:1259] Loaded cluster ID: 338d2ad27f5e4a4f81b54c6dd704849d
I20260806 18:49:22.338351 2532 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b: loading cluster ID for follower catalog manager: success
I20260806 18:49:22.338817 2561 catalog_manager.cc:1259] Loaded cluster ID: 338d2ad27f5e4a4f81b54c6dd704849d
I20260806 18:49:22.338876 2561 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b: loading cluster ID for follower catalog manager: success
I20260806 18:49:22.340246 2532 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b: acquiring CA information for follower catalog manager: success
I20260806 18:49:22.340512 2561 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b: acquiring CA information for follower catalog manager: success
I20260806 18:49:22.341437 2532 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 77ff31328f0e4e318961ab19045ce23b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:49:22.341523 2561 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c403b70667574e5a8c417acdf21c6d5b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:49:22.550076 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:22.546017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52742 (local address 127.1.245.125:35783)
0806 18:49:22.546426 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:22.546431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:22.546447 (+ 16us) server_negotiation.cc:407] Connection header received
0806 18:49:22.546496 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:22.546498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:22.546628 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:22.546750 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:22.547021 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.547594 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.548513 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.548779 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.548863 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:22.549193 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:22.549647 (+ 454us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:22.549739 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:22.549850 (+ 111us) server_negotiation.cc:299] Negotiation successful
0806 18:49:22.549905 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":82,"threads_started":1}
I20260806 18:49:22.556892 2673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:22.553224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41802 (local address 127.1.245.124:36313)
0806 18:49:22.553457 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:22.553461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:22.553475 (+ 14us) server_negotiation.cc:407] Connection header received
0806 18:49:22.553623 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:22.553626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:22.553750 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:22.553853 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:22.554065 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.554639 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.555457 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.555679 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.555771 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:22.556060 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:22.556495 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:22.556606 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:22.556674 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 18:49:22.556739 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":66,"threads_started":1}
I20260806 18:49:22.564345 2662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:22.560406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52748 (local address 127.1.245.125:35783)
0806 18:49:22.560559 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:22.560563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:22.560909 (+ 346us) server_negotiation.cc:407] Connection header received
0806 18:49:22.561111 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:22.561114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:22.561172 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:22.561239 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:22.561643 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.562220 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.563172 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.563499 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.563604 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:22.563889 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:22.563980 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:22.564040 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:22.564132 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 18:49:22.564190 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260806 18:49:22.564670 2673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:22.560168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41818 (local address 127.1.245.124:36313)
0806 18:49:22.560321 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:22.560325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:22.560521 (+ 196us) server_negotiation.cc:407] Connection header received
0806 18:49:22.560721 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:22.560724 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:22.560770 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:22.560836 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:22.561110 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.561792 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.563261 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.563574 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.563659 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:22.563971 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:22.564060 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:22.564139 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:22.564463 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 18:49:22.564528 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260806 18:49:22.564961 2681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:22.560050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44500 (local address 127.1.245.126:38215)
0806 18:49:22.560321 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:22.560326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:22.560592 (+ 266us) server_negotiation.cc:407] Connection header received
0806 18:49:22.560702 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:22.560705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:22.560886 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:22.560998 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:22.561224 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.561834 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.563318 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:22.563558 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:22.563665 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:22.564085 (+ 420us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:22.564491 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:22.564585 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:22.564744 (+ 159us) server_negotiation.cc:299] Negotiation successful
0806 18:49:22.564820 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":90,"threads_started":1}
I20260806 18:49:22.568100 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2184
I20260806 18:49:22.579471 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2284
I20260806 18:49:22.592170 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2382
2026-08-06T18:49:22Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3214 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 18:49:22 dist-test-slave-fgb3 krb5kdc[2685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 18:49:22 dist-test-slave-fgb3 krb5kdc[2685](info): set up 2 sockets
Aug 06 18:49:22 dist-test-slave-fgb3 krb5kdc[2685](info): commencing operation
krb5kdc: starting...
W20260806 18:49:23.703936 2005 mini_kdc.cc:121] Time spent starting KDC: real 1.087s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/test-user.keytab.
Aug 06 18:49:23 dist-test-slave-fgb3 krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T18:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T18:49:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.245.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
Entry for principal kudu/127.1.245.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab.
I20260806 18:49:23.827579 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35595
--webserver_interface=127.1.245.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35595,127.1.245.125:45425,127.1.245.124:36353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 18:49:23.940479 2705 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:49:23.940764 2705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:23.940788 2705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:23.940835 2705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 18:49:23.944830 2705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:23.944892 2705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:23.944911 2705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:23.944926 2705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:23.944952 2705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:23.950282 2705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35595,127.1.245.125:45425,127.1.245.124:36353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.126.keytab
--principal=kudu/127.1.245.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.126:35595
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.245.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:23.951681 2705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:23.952682 2705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:23.960893 2715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:23.960928 2717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:23.960928 2714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:23.961136 2705 server_base.cc:1034] running on GCE node
I20260806 18:49:23.961695 2705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:23.962848 2705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:23.964021 2705 hybrid_clock.cc:648] HybridClock initialized: now 1786042163964002 us; error 33 us; skew 500 ppm
Aug 06 18:49:23 dist-test-slave-fgb3 krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042163, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:23.966799 2705 init.cc:377] Logged in from keytab as kudu/127.1.245.126@KRBTEST.COM (short username kudu)
I20260806 18:49:23.967705 2705 webserver.cc:460] Webserver started at http://127.1.245.126:46661/ using document root <none> and password file <none>
I20260806 18:49:23.968250 2705 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:23.968307 2705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:23.968482 2705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:23.970592 2705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/data/instance:
uuid: "2d8aa6cc565b4470b05770930a901017"
format_stamp: "Formatted at 2026-08-06 18:49:23 on dist-test-slave-fgb3"
server_key: "e2f8760484ed0a3f4c2c571ced46803d"
server_key_iv: "7e7302eae2c311beb6c6a6a38f68c674"
server_key_version: "encryptionkey@0"
I20260806 18:49:23.971148 2705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/wal/instance:
uuid: "2d8aa6cc565b4470b05770930a901017"
format_stamp: "Formatted at 2026-08-06 18:49:23 on dist-test-slave-fgb3"
server_key: "e2f8760484ed0a3f4c2c571ced46803d"
server_key_iv: "7e7302eae2c311beb6c6a6a38f68c674"
server_key_version: "encryptionkey@0"
I20260806 18:49:23.975083 2705 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260806 18:49:23.977669 2727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:23.979002 2705 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:49:23.979125 2705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/wal
uuid: "2d8aa6cc565b4470b05770930a901017"
format_stamp: "Formatted at 2026-08-06 18:49:23 on dist-test-slave-fgb3"
server_key: "e2f8760484ed0a3f4c2c571ced46803d"
server_key_iv: "7e7302eae2c311beb6c6a6a38f68c674"
server_key_version: "encryptionkey@0"
I20260806 18:49:23.979233 2705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:23.988130 2705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:23.991077 2705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:23.991256 2705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:23.998425 2705 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.126:35595
I20260806 18:49:23.998744 2796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.126:35595 every 8 connection(s)
I20260806 18:49:23.999549 2705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/data/info.pb
I20260806 18:49:24.002491 2798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:24.003850 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2705
I20260806 18:49:24.003952 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-0/wal/instance
I20260806 18:49:24.004240 2005 external_mini_cluster.cc:1428] Setting key c8d25c2eaec7201566067d36c76caa17
I20260806 18:49:24.006449 2798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.245.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.125@KRBTEST.COM" created.
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042163, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:24.021938 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.008814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46654 (local address 127.1.245.126:35595)
0806 18:49:24.009244 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.009263 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.009524 (+ 261us) server_negotiation.cc:407] Connection header received
0806 18:49:24.010545 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.010549 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.010737 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.010899 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.011750 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.012676 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.013671 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.014035 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.014271 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.016939 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.016947 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.016961 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.016989 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.018788 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.019392 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.019398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.019400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.019516 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.019857 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.019858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.019860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.020289 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.020402 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.020880 (+ 478us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.021096 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":289,"thread_start_us":120,"threads_started":1}
I20260806 18:49:24.021941 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.008814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35595 (local address 127.0.0.1:46654)
0806 18:49:24.009371 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.009409 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.010225 (+ 816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.010918 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.010923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.011357 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.011605 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.011619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.012793 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.012796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.013541 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.013551 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.014228 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.014233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.014439 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.015029 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.015049 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.016776 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.018897 (+ 2121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.018908 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.018920 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.019239 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.019614 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.019617 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.019619 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.019768 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.020410 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.020417 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.020600 (+ 183us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.020799 (+ 199us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.021189 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":125,"spinlock_wait_cycles":22912,"thread_start_us":169,"threads_started":1}
I20260806 18:49:24.024485 2798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:24.025305 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.025053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39766)
0806 18:49:24.025162 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.025234 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260806 18:49:24.025568 2732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
I20260806 18:49:24.026542 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.026251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39780)
0806 18:49:24.026442 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.026488 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 18:49:24.026695 2798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:45425: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab.
Entry for principal kudu/127.1.245.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab.
I20260806 18:49:24.033275 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.125:45425
--webserver_interface=127.1.245.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab
--principal=kudu/127.1.245.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35595,127.1.245.125:45425,127.1.245.124:36353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 18:49:24.075946 2798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } attempt: 1
I20260806 18:49:24.076913 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.076570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39788)
0806 18:49:24.076746 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.076820 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260806 18:49:24.077757 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.077493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39792)
0806 18:49:24.077620 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.077688 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260806 18:49:24.077901 2798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:45425: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
W20260806 18:49:24.144806 2809 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:49:24.145102 2809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:24.145138 2809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:24.145210 2809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:49:24.146075 2798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } attempt: 2
I20260806 18:49:24.146925 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.146633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39804)
0806 18:49:24.146788 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.146848 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:49:24.147665 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.147444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39808)
0806 18:49:24.147561 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.147605 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260806 18:49:24.147786 2798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.125:45425: Network error: Client connection negotiation failed: client connection to 127.1.245.125:45425: connect: Connection refused (error 111)
W20260806 18:49:24.149413 2809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:24.149466 2809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:24.149493 2809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:24.149534 2809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:24.149562 2809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:24.154740 2809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35595,127.1.245.125:45425,127.1.245.124:36353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.125.keytab
--principal=kudu/127.1.245.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.125:45425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.245.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:24.155753 2809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:24.156883 2809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:24.165154 2820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:24.165112 2818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:24.165200 2817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:24.165220 2809 server_base.cc:1034] running on GCE node
I20260806 18:49:24.165884 2809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:24.167001 2809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:24.168198 2809 hybrid_clock.cc:648] HybridClock initialized: now 1786042164168166 us; error 40 us; skew 500 ppm
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042164, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:24.171047 2809 init.cc:377] Logged in from keytab as kudu/127.1.245.125@KRBTEST.COM (short username kudu)
I20260806 18:49:24.171890 2809 webserver.cc:460] Webserver started at http://127.1.245.125:42539/ using document root <none> and password file <none>
I20260806 18:49:24.172401 2809 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:24.172456 2809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:24.172616 2809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:24.174630 2809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/data/instance:
uuid: "911e8da71a414ebba028a604301d0d8a"
format_stamp: "Formatted at 2026-08-06 18:49:24 on dist-test-slave-fgb3"
server_key: "fddaddc5cf3ee9955c29cb9153bb446e"
server_key_iv: "1f9d4b8ebdf8a99c24bd284349d1b364"
server_key_version: "encryptionkey@0"
I20260806 18:49:24.175136 2809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/wal/instance:
uuid: "911e8da71a414ebba028a604301d0d8a"
format_stamp: "Formatted at 2026-08-06 18:49:24 on dist-test-slave-fgb3"
server_key: "fddaddc5cf3ee9955c29cb9153bb446e"
server_key_iv: "1f9d4b8ebdf8a99c24bd284349d1b364"
server_key_version: "encryptionkey@0"
I20260806 18:49:24.178684 2809 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260806 18:49:24.181012 2828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:24.182264 2809 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260806 18:49:24.182407 2809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/wal
uuid: "911e8da71a414ebba028a604301d0d8a"
format_stamp: "Formatted at 2026-08-06 18:49:24 on dist-test-slave-fgb3"
server_key: "fddaddc5cf3ee9955c29cb9153bb446e"
server_key_iv: "1f9d4b8ebdf8a99c24bd284349d1b364"
server_key_version: "encryptionkey@0"
I20260806 18:49:24.182500 2809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:24.196507 2809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:24.199795 2809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:24.199980 2809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:24.206681 2809 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.125:45425
I20260806 18:49:24.206946 2906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.125:45425 every 8 connection(s)
I20260806 18:49:24.207753 2809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/data/info.pb
I20260806 18:49:24.209733 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2809
I20260806 18:49:24.209857 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-1/wal/instance
I20260806 18:49:24.210139 2005 external_mini_cluster.cc:1428] Setting key d7f0f7efe514c3bf7603e1bb79916e44
I20260806 18:49:24.210917 2907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:24.215003 2907 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.1.245.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.245.124@KRBTEST.COM" created.
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042164, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:24.229861 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.217188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46660 (local address 127.1.245.126:35595)
0806 18:49:24.217341 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.217345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.218651 (+ 1306us) server_negotiation.cc:407] Connection header received
0806 18:49:24.219770 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.219774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.219838 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.219912 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.221278 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.221849 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.223197 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.223456 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.223557 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.226116 (+ 2559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.226119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.226121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.226150 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.227692 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.228226 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.228229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.228230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.228312 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.228664 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.228667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.228669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.228864 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.228968 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.229540 (+ 572us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.229686 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260806 18:49:24.230819 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.217777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35595 (local address 127.0.0.1:46660)
0806 18:49:24.218481 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.218534 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.219525 (+ 991us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.220111 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.220119 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.220493 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.221110 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.221126 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.221989 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.221992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.223053 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.223066 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.223630 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.223634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.223848 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.224479 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.224501 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.225930 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.227825 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.227832 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.227850 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.228123 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.228419 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.228422 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.228424 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.228573 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.228982 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.228990 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.229274 (+ 284us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.229465 (+ 191us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.229824 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":207,"threads_started":1}
I20260806 18:49:24.232309 2907 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.245.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab.
Entry for principal kudu/127.1.245.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab.
I20260806 18:49:24.237471 2005 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskesmHnt/build/debug/bin/kudu
/tmp/dist-test-taskesmHnt/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.124:36353
--webserver_interface=127.1.245.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab
--principal=kudu/127.1.245.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.245.84:37901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.245.126:35595,127.1.245.125:45425,127.1.245.124:36353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042164, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
I20260806 18:49:24.238927 2798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } attempt: 3
I20260806 18:49:24.241811 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.232833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39822)
0806 18:49:24.232967 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.232979 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.233055 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.233643 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.233646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.233823 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.233917 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.233922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.234959 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.234962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.235542 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.235551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.236155 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.236159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.236247 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.236760 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.236774 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.237980 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.240157 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.240160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.240162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.240383 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.240676 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.240679 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.240681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.240792 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.241297 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.241300 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.241401 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.241505 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.241629 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260806 18:49:24.242017 2914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.232927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39822 (local address 127.1.245.125:45425)
0806 18:49:24.233281 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.233296 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.233336 (+ 40us) server_negotiation.cc:407] Connection header received
0806 18:49:24.233371 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.233376 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.233529 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.233644 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.234017 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.234830 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.235650 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.236051 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.236143 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.238109 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.238118 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.238130 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.238151 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.240041 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.240484 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.240489 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.240490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.240582 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.240881 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.240884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.240894 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.241186 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.241324 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.241616 (+ 292us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.241788 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":132,"threads_started":1}
I20260806 18:49:24.244552 2907 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:24.245769 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.244985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44152)
0806 18:49:24.245593 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.245682 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529}
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042163, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
W20260806 18:49:24.245970 2834 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
I20260806 18:49:24.246974 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.246735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44166)
0806 18:49:24.246847 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.246908 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 18:49:24.247391 2907 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36353: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
I20260806 18:49:24.249686 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.239412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39836)
0806 18:49:24.239539 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.239554 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.239641 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.240168 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.240171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.240337 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.240448 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.240456 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.241367 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.241370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.242685 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.242698 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.244387 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.244391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.244511 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.244948 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.244964 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.246301 (+ 1337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.248304 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.248306 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.248308 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.248507 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.248809 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.248812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.248813 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.248899 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.249300 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.249302 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.249380 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.249461 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.249571 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260806 18:49:24.250006 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.239611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39836 (local address 127.1.245.125:45425)
0806 18:49:24.239951 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.239957 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.239968 (+ 11us) server_negotiation.cc:407] Connection header received
0806 18:49:24.240005 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.240007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.240041 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.240116 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.240573 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.241245 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.242912 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.243235 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.243335 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.246491 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.246494 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.246497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.246520 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.248197 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.248601 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.248604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.248606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.248701 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.248981 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.248985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.248987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.249216 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.249326 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.249538 (+ 212us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.249661 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":140,"threads_started":1}
I20260806 18:49:24.250711 2798 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } has no permanent_uuid. Determining permanent_uuid...
I20260806 18:49:24.251335 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.251122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44182)
0806 18:49:24.251227 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.251277 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 18:49:24.252009 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.251776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44198)
0806 18:49:24.251921 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.251962 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260806 18:49:24.252139 2798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36353: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
I20260806 18:49:24.283313 2798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } attempt: 1
I20260806 18:49:24.284220 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.283886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44214)
0806 18:49:24.284056 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.284122 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260806 18:49:24.284925 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.284697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44216)
0806 18:49:24.284808 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.284856 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 18:49:24.285087 2798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36353: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
I20260806 18:49:24.296687 2907 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } attempt: 1
I20260806 18:49:24.297631 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.297319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44222)
0806 18:49:24.297478 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.297549 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 18:49:24.298532 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.298284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44234)
0806 18:49:24.298420 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.298471 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260806 18:49:24.298657 2907 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36353: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
W20260806 18:49:24.359109 2917 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 18:49:24.359396 2917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 18:49:24.359443 2917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 18:49:24.359506 2917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260806 18:49:24.360270 2798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } attempt: 2
I20260806 18:49:24.361104 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.360799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44238)
0806 18:49:24.360958 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.361021 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260806 18:49:24.361912 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.361676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44250)
0806 18:49:24.361784 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.361841 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260806 18:49:24.362046 2798 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36353: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
W20260806 18:49:24.363969 2917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 18:49:24.364032 2917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 18:49:24.364051 2917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 18:49:24.364080 2917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 18:49:24.364095 2917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 18:49:24.366808 2907 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } attempt: 2
I20260806 18:49:24.367542 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.367282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44256)
0806 18:49:24.367415 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.367471 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 18:49:24.368199 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.367980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44262)
0806 18:49:24.368073 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.368139 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260806 18:49:24.368315 2907 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.245.124:36353: Network error: Client connection negotiation failed: client connection to 127.1.245.124:36353: connect: Connection refused (error 111)
I20260806 18:49:24.369741 2917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.245.84:37901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.245.126:35595,127.1.245.125:45425,127.1.245.124:36353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/krb5kdc/kudu_127.1.245.124.keytab
--principal=kudu/127.1.245.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.245.124:36353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.245.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 17:59:13 UTC on ccycloud.quasar-wvsnzx.root.comops.site
build id 4571028
I20260806 18:49:24.370888 2917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 18:49:24.371901 2917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 18:49:24.380716 2926 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 18:49:24.380856 2929 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:24.380771 2917 server_base.cc:1034] running on GCE node
W20260806 18:49:24.380736 2927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 18:49:24.381670 2917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 18:49:24.382983 2917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 18:49:24.384188 2917 hybrid_clock.cc:648] HybridClock initialized: now 1786042164384161 us; error 37 us; skew 500 ppm
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042164, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 18:49:24.387220 2917 init.cc:377] Logged in from keytab as kudu/127.1.245.124@KRBTEST.COM (short username kudu)
I20260806 18:49:24.388231 2917 webserver.cc:460] Webserver started at http://127.1.245.124:40137/ using document root <none> and password file <none>
I20260806 18:49:24.388810 2917 fs_manager.cc:359] Metadata directory not provided
I20260806 18:49:24.388872 2917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 18:49:24.389091 2917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 18:49:24.391326 2917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/data/instance:
uuid: "0b6e716f62574324be2bfa35fbb7d664"
format_stamp: "Formatted at 2026-08-06 18:49:24 on dist-test-slave-fgb3"
server_key: "6b6e9771fe59e77ca9ee7f216a4f1297"
server_key_iv: "e71c111223204ce9d51caa45b3fae4a7"
server_key_version: "encryptionkey@0"
I20260806 18:49:24.391920 2917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/wal/instance:
uuid: "0b6e716f62574324be2bfa35fbb7d664"
format_stamp: "Formatted at 2026-08-06 18:49:24 on dist-test-slave-fgb3"
server_key: "6b6e9771fe59e77ca9ee7f216a4f1297"
server_key_iv: "e71c111223204ce9d51caa45b3fae4a7"
server_key_version: "encryptionkey@0"
I20260806 18:49:24.395831 2917 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260806 18:49:24.398435 2937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 18:49:24.399747 2917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260806 18:49:24.399878 2917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/data,/tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/wal
uuid: "0b6e716f62574324be2bfa35fbb7d664"
format_stamp: "Formatted at 2026-08-06 18:49:24 on dist-test-slave-fgb3"
server_key: "6b6e9771fe59e77ca9ee7f216a4f1297"
server_key_iv: "e71c111223204ce9d51caa45b3fae4a7"
server_key_version: "encryptionkey@0"
I20260806 18:49:24.399991 2917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 18:49:24.410768 2917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 18:49:24.413518 2917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 18:49:24.413687 2917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 18:49:24.420657 2917 rpc_server.cc:307] RPC server started. Bound to: 127.1.245.124:36353
I20260806 18:49:24.420922 3013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.245.124:36353 every 8 connection(s)
I20260806 18:49:24.421625 2917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/data/info.pb
I20260806 18:49:24.424469 3014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 18:49:24.425366 2005 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskesmHnt/build/debug/bin/kudu as pid 2917
I20260806 18:49:24.425501 2005 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskesmHnt/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786041874296929-2005-0/minicluster-data/master-2/wal/instance
I20260806 18:49:24.425817 2005 external_mini_cluster.cc:1428] Setting key 4144bd5bd473cd5683c4550b406538bd
I20260806 18:49:24.428761 3014 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042164, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:24.440213 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.430686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46662 (local address 127.1.245.126:35595)
0806 18:49:24.430794 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.430797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.431135 (+ 338us) server_negotiation.cc:407] Connection header received
0806 18:49:24.431286 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.431289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.431346 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.431420 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.432222 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.432970 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.434094 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.434332 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.434425 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.436982 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.436985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.436988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.437019 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.438471 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.438892 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.438903 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.438905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.438990 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.439280 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.439283 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.439284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.439514 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.439606 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.439987 (+ 381us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.440090 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:49:24.440773 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.427173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.1.245.124:36353)
0806 18:49:24.427564 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.427580 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.427616 (+ 36us) server_negotiation.cc:407] Connection header received
0806 18:49:24.428262 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.428279 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.428627 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.428906 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.429777 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.431077 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.432199 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.432913 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.433080 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.435801 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.435809 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.435823 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.435850 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.437541 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.438030 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.438037 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.438045 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.438176 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.438450 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.438454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.438455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.438951 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.439165 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.439438 (+ 273us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.439740 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"spinlock_wait_cycles":2816,"thread_start_us":99,"threads_started":1}
I20260806 18:49:24.440773 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.430641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.126:35595 (local address 127.0.0.1:46662)
0806 18:49:24.431011 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.431050 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.431168 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.431498 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.431504 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.431944 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.432099 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.432106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.433096 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.433099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.433964 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.433976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.434505 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.434510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.434627 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.435125 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.435141 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.436825 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.438568 (+ 1743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.438576 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.438582 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.438800 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.439085 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.439087 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.439089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.439184 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.439656 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.439663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.439775 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.439927 (+ 152us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.440085 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":139,"threads_started":1}
I20260806 18:49:24.442026 3014 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042164, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.126@KRBTEST.COM
I20260806 18:49:24.450230 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.442540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.125:45425 (local address 127.0.0.1:39850)
0806 18:49:24.442845 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.442858 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.442942 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.443275 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.443279 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.443416 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.443490 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.443494 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.444251 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.444255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.444754 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.444761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.445206 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.445209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.445288 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.445697 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.445711 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.447017 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.448735 (+ 1718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.448738 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.448741 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.448927 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.449201 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.449204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.449205 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.449283 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.449691 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.449694 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.449792 (+ 98us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.449928 (+ 136us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.450061 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247}
I20260806 18:49:24.450264 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.442667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39850 (local address 127.1.245.125:45425)
0806 18:49:24.442823 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.442827 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.442896 (+ 69us) server_negotiation.cc:407] Connection header received
0806 18:49:24.443031 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.443035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.443084 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.443154 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.443604 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.444153 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.444856 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.445109 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.445197 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.447163 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.447166 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.447168 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.447192 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.448636 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.449012 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.449015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.449017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.449102 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.449367 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.449371 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.449373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.449595 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.449697 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.449973 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.450104 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260806 18:49:24.450557 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.443178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46674 (local address 127.1.245.126:35595)
0806 18:49:24.443282 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.443285 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.443382 (+ 97us) server_negotiation.cc:407] Connection header received
0806 18:49:24.443477 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.443480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.443520 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.443581 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.444148 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.444622 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.445329 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.445547 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.445631 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.447787 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.447790 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.447792 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.447813 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.449014 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.449400 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.449404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.449405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.449472 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.449709 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.449711 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.449713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.449926 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.450139 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.450258 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.450400 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:49:24.451196 3014 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042164, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.124@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.245.125@KRBTEST.COM
I20260806 18:49:24.458998 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.451700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44284)
0806 18:49:24.451920 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.451929 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.452004 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.452183 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.452185 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.452328 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.452389 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.452393 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.453421 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.453424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.453902 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.453908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.454296 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.454299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.454445 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.454890 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.454903 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.456028 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.457582 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.457585 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.457587 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.457763 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.458007 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.458010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.458012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.458091 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.458506 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.458508 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.458599 (+ 91us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.458706 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.458853 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":52}
I20260806 18:49:24.459108 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.451742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44284 (local address 127.1.245.124:36353)
0806 18:49:24.451874 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.451878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.451982 (+ 104us) server_negotiation.cc:407] Connection header received
0806 18:49:24.452067 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.452070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.452106 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.452163 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.452490 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.453318 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.453996 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.454209 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.454288 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.456160 (+ 1872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.456165 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.456168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.456194 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.457485 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.457845 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.457848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.457850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.457918 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.458174 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.458177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.458178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.458407 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.458500 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.458753 (+ 253us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.458921 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":63}
I20260806 18:49:24.459442 2907 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } attempt: 3
I20260806 18:49:24.459837 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.451796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39864 (local address 127.1.245.125:45425)
0806 18:49:24.451905 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.451909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.451972 (+ 63us) server_negotiation.cc:407] Connection header received
0806 18:49:24.452094 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.452097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.452137 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.452199 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.452748 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.453234 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.453944 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.454161 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.454245 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.456608 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.456612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.456614 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.456640 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.458010 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.458433 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.458437 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.458438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.458526 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.458781 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.458784 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.458786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.459014 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.459246 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.459571 (+ 325us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.459686 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 18:49:24.461277 2798 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } attempt: 3
I20260806 18:49:24.462838 3014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664: Bootstrap starting.
I20260806 18:49:24.464016 2005 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042164, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.125@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
I20260806 18:49:24.465987 3014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:24.465262 2762 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "2d8aa6cc565b4470b05770930a901017"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46674
I20260806 18:49:24.466953 3014 log.cc:824] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664: Log is configured to *not* fsync() on all Append() calls
Aug 06 18:49:24 dist-test-slave-fgb3 krb5kdc[2685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786042163, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.245.126@KRBTEST.COM for kudu/127.1.245.124@KRBTEST.COM
I20260806 18:49:24.469442 2910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.459841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44288)
0806 18:49:24.460269 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.460283 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.460363 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.460606 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.460609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.460787 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.460871 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.460876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.461741 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.461745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.462490 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.462502 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.463058 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.463062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.463148 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.463624 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.463638 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.465113 (+ 1475us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.467772 (+ 2659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.467776 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.467779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.467988 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.468277 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.468280 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.468282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.468381 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.468923 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.468926 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.469028 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.469125 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.469268 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353}
I20260806 18:49:24.469462 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.460249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44288 (local address 127.1.245.124:36353)
0806 18:49:24.460344 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.460353 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.460365 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:49:24.460441 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.460444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.460486 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.460977 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.461064 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.461625 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.462623 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.462933 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.463358 (+ 425us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.465242 (+ 1884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.465246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.465249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.465274 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.467643 (+ 2369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.468080 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.468083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.468084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.468186 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.468567 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.468571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.468573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.468818 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.468912 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.469191 (+ 279us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.469341 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 18:49:24.470278 3014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664: No bootstrap required, opened a new log
I20260806 18:49:24.471366 2806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.461723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.245.124:36353 (local address 127.0.0.1:44290)
0806 18:49:24.461910 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 18:49:24.461924 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 18:49:24.461999 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 18:49:24.462758 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 18:49:24.462761 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 18:49:24.462957 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0806 18:49:24.463067 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.463075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.464262 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.464266 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.465079 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 18:49:24.465091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.465741 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.465745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 18:49:24.465832 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.466255 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 18:49:24.466269 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0806 18:49:24.467900 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 18:49:24.469939 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.469943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.469945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.470184 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.470478 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 18:49:24.470481 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 18:49:24.470482 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 18:49:24.470561 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 18:49:24.470899 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 18:49:24.470902 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 18:49:24.470991 (+ 89us) client_negotiation.cc:769] Sending connection context
0806 18:49:24.471075 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 18:49:24.471206 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114}
I20260806 18:49:24.471406 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.462175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44290 (local address 127.1.245.124:36353)
0806 18:49:24.462531 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.462540 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.462552 (+ 12us) server_negotiation.cc:407] Connection header received
0806 18:49:24.462583 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.462585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.462628 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.463226 (+ 598us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.463316 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.464130 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.465292 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.465644 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.465737 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.468030 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.468034 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.468036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.468060 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.469798 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.470278 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.470282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.470284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.470394 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.470640 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.470643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.470645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.470803 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.470886 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.471135 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.471235 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260806 18:49:24.472615 2907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a: Bootstrap starting.
I20260806 18:49:24.473634 2798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017: Bootstrap starting.
I20260806 18:49:24.474736 3014 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } }
I20260806 18:49:24.475066 3014 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b6e716f62574324be2bfa35fbb7d664, State: Initialized, Role: FOLLOWER
I20260806 18:49:24.475404 2907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:24.475596 2798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017: Neither blocks nor log segments found. Creating new log.
I20260806 18:49:24.475534 3014 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } }
I20260806 18:49:24.476174 3026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.476312 3026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:24.476291 2907 log.cc:824] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:24.476302 2798 log.cc:824] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017: Log is configured to *not* fsync() on all Append() calls
I20260806 18:49:24.477188 3014 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:24.478492 2798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017: No bootstrap required, opened a new log
I20260806 18:49:24.479691 2907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a: No bootstrap required, opened a new log
I20260806 18:49:24.481449 2798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } }
I20260806 18:49:24.481673 2798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d8aa6cc565b4470b05770930a901017, State: Initialized, Role: FOLLOWER
I20260806 18:49:24.481994 2798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } }
I20260806 18:49:24.483379 2798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:24.483489 2762 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 18:49:24.483604 2762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:24.483531 2907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } }
I20260806 18:49:24.483727 2907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 911e8da71a414ebba028a604301d0d8a, State: Initialized, Role: FOLLOWER
I20260806 18:49:24.484012 2907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } }
I20260806 18:49:24.480866 2917 master_runner.cc:186] Error getting master registration for 127.1.245.126:35595: OK, instance_id { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" instance_seqno: 1786042163996333 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 18:49:24.484726 2917 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:49:24.484877 2762 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } }
I20260806 18:49:24.485344 2907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 18:49:24.485855 2762 leader_election.cc:288] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [CANDIDATE]: Term 1 election: Requested vote from peers 911e8da71a414ebba028a604301d0d8a (127.1.245.125:45425), 0b6e716f62574324be2bfa35fbb7d664 (127.1.245.124:36353)
I20260806 18:49:24.487617 2975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d8aa6cc565b4470b05770930a901017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0b6e716f62574324be2bfa35fbb7d664"
I20260806 18:49:24.487849 2975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [term 0 FOLLOWER]: Advancing to term 1
W20260806 18:49:24.489123 3063 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:24.489188 3063 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:24.489180 2975 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d8aa6cc565b4470b05770930a901017 in term 1.
I20260806 18:49:24.489938 2870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2d8aa6cc565b4470b05770930a901017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "911e8da71a414ebba028a604301d0d8a"
I20260806 18:49:24.490147 2870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [term 0 FOLLOWER]: Advancing to term 1
I20260806 18:49:24.491262 2733 leader_election.cc:302] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b6e716f62574324be2bfa35fbb7d664, 2d8aa6cc565b4470b05770930a901017; no voters:
I20260806 18:49:24.491688 3088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.491961 3088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:24.492098 3088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 18:49:24.492223 3088 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [term 1 LEADER]: Becoming Leader. State: Replica: 2d8aa6cc565b4470b05770930a901017, State: Running, Role: LEADER
I20260806 18:49:24.492597 3088 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } }
W20260806 18:49:24.493012 2735 tablet.cc:2369] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 18:49:24.493976 2870 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d8aa6cc565b4470b05770930a901017 in term 1.
I20260806 18:49:24.495285 3088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d8aa6cc565b4470b05770930a901017. Latest consensus state: current_term: 1 leader_uuid: "2d8aa6cc565b4470b05770930a901017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.495462 3088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: This master's current role is: LEADER
W20260806 18:49:24.497383 3081 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:24.497463 3081 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:24.502486 2809 master_runner.cc:263] No leader master found from master 911e8da71a414ebba028a604301d0d8a
I20260806 18:49:24.502562 2809 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 18:49:24.505143 2705 master_runner.cc:263] No leader master found from master 2d8aa6cc565b4470b05770930a901017
I20260806 18:49:24.505211 2705 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260806 18:49:24.506392 3046 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 18:49:24.506465 3046 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 18:49:24.506699 3050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.506830 3050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:24.511812 2918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.494749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39866 (local address 127.1.245.125:45425)
0806 18:49:24.494868 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.494872 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.502627 (+ 7755us) server_negotiation.cc:407] Connection header received
0806 18:49:24.502814 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.502817 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.502865 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.502915 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.504135 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.504833 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.506025 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.506350 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.506440 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.508329 (+ 1889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.508333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.508335 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.508363 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.509986 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.510464 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.510468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.510469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.510555 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.510869 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.510872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.510874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.511098 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.511203 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.511549 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.511659 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:49:24.515058 3092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 18:49:24.516137 3092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 18:49:24.520131 3024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.495553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.1.245.124:36353)
0806 18:49:24.495668 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.495673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.502855 (+ 7182us) server_negotiation.cc:407] Connection header received
0806 18:49:24.503036 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.503039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.503088 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.503138 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.507195 (+ 4057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.507952 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.510276 (+ 2324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.510655 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.510734 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.515253 (+ 4519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.515257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.515259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.515285 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.517006 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.518488 (+ 1482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.518492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.518493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.518588 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.519143 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.519147 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.519149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.519380 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.519512 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.519856 (+ 344us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.519977 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 18:49:24.521750 2802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:24.501425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46684 (local address 127.1.245.126:35595)
0806 18:49:24.501534 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.501539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.509198 (+ 7659us) server_negotiation.cc:407] Connection header received
0806 18:49:24.509375 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.509379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.509447 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.509500 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.511297 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.511979 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.514214 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:24.514541 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:24.514612 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:24.518483 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 18:49:24.518487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 18:49:24.518489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 18:49:24.518515 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0806 18:49:24.520009 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.520461 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.520464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.520466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.520549 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 18:49:24.520839 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 18:49:24.520842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 18:49:24.520844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0806 18:49:24.521068 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 18:49:24.521173 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:24.521506 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 18:49:24.521620 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 18:49:24.533558 2975 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [term 1 FOLLOWER]: Refusing update from remote peer 2d8aa6cc565b4470b05770930a901017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:49:24.534246 3048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:49:24.538291 3026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d8aa6cc565b4470b05770930a901017. Latest consensus state: current_term: 1 leader_uuid: "2d8aa6cc565b4470b05770930a901017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.538427 3026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:24.539425 2870 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [term 1 FOLLOWER]: Refusing update from remote peer 2d8aa6cc565b4470b05770930a901017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 18:49:24.540030 3048 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 18:49:24.543936 3050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d8aa6cc565b4470b05770930a901017. Latest consensus state: current_term: 1 leader_uuid: "2d8aa6cc565b4470b05770930a901017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.544076 3050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:24.550062 3048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d8aa6cc565b4470b05770930a901017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.550218 3048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:24.551512 3050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d8aa6cc565b4470b05770930a901017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.551631 3050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:24.552002 3102 mvcc.cc:205] Tried to move back new op lower bound from 7315628705921753088 to 7315628705768194048. Current Snapshot: MvccSnapshot[applied={T|T < 7315628705921753088}]
I20260806 18:49:24.554698 3088 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2d8aa6cc565b4470b05770930a901017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.554812 3088 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017 [sys.catalog]: This master's current role is: LEADER
I20260806 18:49:24.556262 3026 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2d8aa6cc565b4470b05770930a901017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2d8aa6cc565b4470b05770930a901017" member_type: VOTER last_known_addr { host: "127.1.245.126" port: 35595 } } peers { permanent_uuid: "911e8da71a414ebba028a604301d0d8a" member_type: VOTER last_known_addr { host: "127.1.245.125" port: 45425 } } peers { permanent_uuid: "0b6e716f62574324be2bfa35fbb7d664" member_type: VOTER last_known_addr { host: "127.1.245.124" port: 36353 } } }
I20260806 18:49:24.556384 3026 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 18:49:24.557442 3106 mvcc.cc:205] Tried to move back new op lower bound from 7315628705921753088 to 7315628705768194048. Current Snapshot: MvccSnapshot[applied={T|T < 7315628705921753088}]
I20260806 18:49:24.557621 3103 mvcc.cc:205] Tried to move back new op lower bound from 7315628705921753088 to 7315628705768194048. Current Snapshot: MvccSnapshot[applied={T|T < 7315628705921753088}]
I20260806 18:49:24.562525 3092 catalog_manager.cc:1347] Generated new cluster ID: cfd84bde09f946a687b1e34540a5b139
I20260806 18:49:24.562592 3092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 18:49:24.584473 3092 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 18:49:24.585906 3092 catalog_manager.cc:1504] Loading token signing keys...
I20260806 18:49:24.602147 3092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d8aa6cc565b4470b05770930a901017: Generated new TSK 0
I20260806 18:49:24.603117 3092 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 18:49:24.637646 3024 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.634609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.1.245.124:36353)
0806 18:49:24.634733 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.634738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.636604 (+ 1866us) server_negotiation.cc:407] Connection header received
0806 18:49:24.636791 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.636795 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.636844 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.636904 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.637511 (+ 607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44312: BlockingRecv error: recv got EOF from 127.0.0.1:44312 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260806 18:49:24.639629 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.635345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.1.245.125:45425)
0806 18:49:24.635459 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.635464 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.638202 (+ 2738us) server_negotiation.cc:407] Connection header received
0806 18:49:24.638604 (+ 402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.638607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.638657 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.638706 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.639509 (+ 803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39882: BlockingRecv error: recv got EOF from 127.0.0.1:39882 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260806 18:49:24.640355 2802 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.636040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46698 (local address 127.1.245.126:35595)
0806 18:49:24.636149 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.636153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.638376 (+ 2223us) server_negotiation.cc:407] Connection header received
0806 18:49:24.638845 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.638849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.638918 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.638970 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.640237 (+ 1267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46698: BlockingRecv error: recv got EOF from 127.0.0.1:46698 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260806 18:49:24.644992 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.643494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39894 (local address 127.1.245.125:45425)
0806 18:49:24.643598 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.643601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.644075 (+ 474us) server_negotiation.cc:407] Connection header received
0806 18:49:24.644274 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.644278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.644351 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.644402 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.644881 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39894: BlockingRecv error: recv got EOF from 127.0.0.1:39894 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 18:49:24.650390 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.648791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39900 (local address 127.1.245.125:45425)
0806 18:49:24.648903 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.648907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.649431 (+ 524us) server_negotiation.cc:407] Connection header received
0806 18:49:24.649618 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.649622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.649672 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.649721 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.650227 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39900: BlockingRecv error: recv got EOF from 127.0.0.1:39900 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 18:49:24.655812 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.654355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39902 (local address 127.1.245.125:45425)
0806 18:49:24.654456 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.654460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.654954 (+ 494us) server_negotiation.cc:407] Connection header received
0806 18:49:24.655134 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.655138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.655185 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.655232 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.655712 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39902: BlockingRecv error: recv got EOF from 127.0.0.1:39902 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260806 18:49:24.663725 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.662029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.1.245.125:45425)
0806 18:49:24.662169 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.662174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.662754 (+ 580us) server_negotiation.cc:407] Connection header received
0806 18:49:24.662963 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.662967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.663021 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.663073 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.663615 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39910: BlockingRecv error: recv got EOF from 127.0.0.1:39910 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260806 18:49:24.679260 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.674095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39920 (local address 127.1.245.125:45425)
0806 18:49:24.674226 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.674231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.674920 (+ 689us) server_negotiation.cc:407] Connection header received
0806 18:49:24.675157 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.675161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.675222 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.675282 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.679081 (+ 3799us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39920: BlockingRecv error: recv got EOF from 127.0.0.1:39920 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260806 18:49:24.700582 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.699247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39932 (local address 127.1.245.125:45425)
0806 18:49:24.699430 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.699434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.699550 (+ 116us) server_negotiation.cc:407] Connection header received
0806 18:49:24.699830 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.699834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.699893 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.699968 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.700455 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39932: BlockingRecv error: recv got EOF from 127.0.0.1:39932 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260806 18:49:24.736079 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.734848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.1.245.125:45425)
0806 18:49:24.735030 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.735035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.735136 (+ 101us) server_negotiation.cc:407] Connection header received
0806 18:49:24.735297 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.735301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.735354 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.735433 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.735945 (+ 512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39934: BlockingRecv error: recv got EOF from 127.0.0.1:39934 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260806 18:49:24.803443 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.802183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.1.245.125:45425)
0806 18:49:24.802393 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.802398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.802493 (+ 95us) server_negotiation.cc:407] Connection header received
0806 18:49:24.802658 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.802662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.802716 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.802787 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.803303 (+ 516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39950: BlockingRecv error: recv got EOF from 127.0.0.1:39950 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260806 18:49:24.934957 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:24.933772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39958 (local address 127.1.245.125:45425)
0806 18:49:24.933924 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:24.933929 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:24.934032 (+ 103us) server_negotiation.cc:407] Connection header received
0806 18:49:24.934193 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:24.934196 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:24.934250 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:24.934356 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:24.934829 (+ 473us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39958: BlockingRecv error: recv got EOF from 127.0.0.1:39958 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260806 18:49:25.194206 2918 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 18:49:25.193138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39964 (local address 127.1.245.125:45425)
0806 18:49:25.193259 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:25.193262 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:25.193445 (+ 183us) server_negotiation.cc:407] Connection header received
0806 18:49:25.193519 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:25.193521 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:25.193577 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:25.193660 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 18:49:25.194080 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39964: BlockingRecv error: recv got EOF from 127.0.0.1:39964 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 18:49:25.498987 3081 catalog_manager.cc:1259] Loaded cluster ID: cfd84bde09f946a687b1e34540a5b139
I20260806 18:49:25.499084 3081 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a: loading cluster ID for follower catalog manager: success
I20260806 18:49:25.500830 3081 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a: acquiring CA information for follower catalog manager: success
I20260806 18:49:25.502122 3081 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 911e8da71a414ebba028a604301d0d8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:49:25.507755 3046 catalog_manager.cc:1259] Loaded cluster ID: cfd84bde09f946a687b1e34540a5b139
I20260806 18:49:25.507827 3046 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664: loading cluster ID for follower catalog manager: success
I20260806 18:49:25.509537 3046 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664: acquiring CA information for follower catalog manager: success
I20260806 18:49:25.510747 3046 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0b6e716f62574324be2bfa35fbb7d664: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 18:49:25.712219 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:25.708510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.1.245.125:45425)
0806 18:49:25.708736 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:25.708739 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:25.708754 (+ 15us) server_negotiation.cc:407] Connection header received
0806 18:49:25.708905 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:25.708908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:25.709039 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:25.709149 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:25.709392 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.709923 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.710841 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.711088 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.711169 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:25.711438 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:25.711871 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:25.711976 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:25.712039 (+ 63us) server_negotiation.cc:299] Negotiation successful
0806 18:49:25.712092 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":74,"threads_started":1}
I20260806 18:49:25.718730 3215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:25.715194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44326 (local address 127.1.245.124:36353)
0806 18:49:25.715429 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:25.715433 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:25.715452 (+ 19us) server_negotiation.cc:407] Connection header received
0806 18:49:25.715531 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:25.715533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:25.715650 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:25.715742 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:25.715956 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.716484 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.717312 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.717542 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.717633 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:25.717870 (+ 237us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:25.718361 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:25.718464 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:25.718519 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 18:49:25.718570 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":63,"threads_started":1}
I20260806 18:49:25.725534 3215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:25.721630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44328 (local address 127.1.245.124:36353)
0806 18:49:25.721787 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:25.721811 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:25.722300 (+ 489us) server_negotiation.cc:407] Connection header received
0806 18:49:25.722372 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:25.722375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:25.722431 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:25.722490 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:25.722721 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.723255 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.724225 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.724563 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.724669 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:25.724895 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:25.724975 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:25.725033 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:25.725334 (+ 301us) server_negotiation.cc:299] Negotiation successful
0806 18:49:25.725388 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260806 18:49:25.725715 3205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:25.721650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39970 (local address 127.1.245.125:45425)
0806 18:49:25.721779 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:25.721783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:25.722189 (+ 406us) server_negotiation.cc:407] Connection header received
0806 18:49:25.722382 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:25.722385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:25.722434 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:25.722516 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:25.722855 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.723619 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.724615 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.724933 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.725025 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:25.725243 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:25.725343 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:25.725405 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:25.725534 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 18:49:25.725599 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 18:49:25.726429 3221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 18:49:25.721552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46710 (local address 127.1.245.126:35595)
0806 18:49:25.721787 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0806 18:49:25.721792 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 18:49:25.722034 (+ 242us) server_negotiation.cc:407] Connection header received
0806 18:49:25.722247 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 18:49:25.722250 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 18:49:25.722432 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 18:49:25.722540 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 18:49:25.722790 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.723617 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.724778 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 18:49:25.725100 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 18:49:25.725170 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 18:49:25.725600 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 18:49:25.725978 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 18:49:25.726070 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 18:49:25.726187 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 18:49:25.726253 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":72,"threads_started":1}
I20260806 18:49:25.729357 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2705
I20260806 18:49:25.744119 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2809
I20260806 18:49:25.755019 2005 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskesmHnt/build/debug/bin/kudu with pid 2917
2026-08-06T18:49:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3163 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6378 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291452 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST